Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “prediction uncertainty”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Selection of high-redshift Lyman-Break Galaxies from broadband and wide photometric surveys

In this paper, we investigate the possibility of selecting high-redshift Lyman-Break Galaxies (LBG) using current and future broadband wide photometric surveys, such as the Ultraviolet Near Infrared Optical Northern Survey (UNIONS) or the Vera C. Rubin Legacy Survey of Space and Time (LSST), using a Random Forest algorithm. This work is conducted in the context of future large-scale structure spectroscopic surveys like DESI-II, the next phase of the Dark Energy Spectroscopic Instrument (DESI), which will start around 2029.We use deep imaging data from the Hyper Suprime Camera (HSC) and the Canada-France-Hawaii Telescope Large Area U-band Deep Survey (CLAUDS) on the COSMOS and XMM-LSS fields. To predict the selection performance of LBGs with image quality similar to UNIONS, we degrade the u,g,r,i and z bands to UNIONS depth.The Random Forest algorithm is trained with the u,g,r,i and z bands to classify LBGs in the 2.5 < z < 3.5 range.We find that fixing a target density budget of 1,100 deg$^{-2}$, the Random Forest approach gives a density of z > 2 targets of 873 deg$^{-2}$, and a density of 493 deg$^{-2}$ of confirmed LBGs after spectroscopic confirmation with DESI. This UNIONS-like selection was tested in a dedicated spectroscopic observation campaign of 1,000 targets with DESI on the COSMOS field, providing a safe spectroscopic sample with a mean redshift of 3. This sample is used to derive forecasts for DESI-II, assuming a sky coverage of 5,000 deg$^{2}$. We predict uncertainties on Alcock-Paczynski parameters α$_{⊥}$ and α$_{∥}$ to be 0.7% and 1% for 2.6 < z < 3.2, resulting in a potential 2% measurement of the dark energy fraction at high redshift. Additionally, we estimate the uncertainty in local non-Gaussianity and predict σ$_{fNL}$ ≈ 7, which would be comparable to the current best precision achieved by Planck. The latter forecast suggests that achieving the precision required to place stringent constraints on inflationary models (σ$_{fNL}$ ≈ 1) using spectroscopic galaxy surveys necessitates the development of a next-generation (Stage V) spectroscopic survey.

79 ASTRONOMY AND ASTROPHYSICS↗

GPLaSDI: Gaussian Process Latent Space Dynamics Identification

GPLaSDI is a software that introduces GPLaSDI, a novel LaSDI-based framework that relies on Gaussian process (GP) for latent space ODE interpolations. Using GPs offers two significant advantages. First, it enables the quantification of uncertainty over the reduced order model (ROM) predictions. Second, leveraging this prediction uncertainty allows for efficient adaptive training through a greedy selection of additional training data points. This approach does not require prior knowledge of the underlying PDEs. Consequently, GPLaSDI is inherently non-intrusive and can be applied to problems without a known PDE or its residual. We demonstrate the effectiveness of our approach on the Burgers equation, Vlasov equation for plasma physics, and a rising thermal bubble problem. Our proposed method achieves between 200 and 100,000 times speed-up, with up to 7% relative error.

Choi, Youngsoo↗

Deep Direct-Use Feasibility Study Numerical Modeling and Uncertainty Analysis using iTOUGH2 for West Virginia University

To reduce the geothermal exploration risk, a feasibility study is performed for a deep direct-use system proposed at the West Virginia University (WVU) Morgantown campus. This study applies numerical simulations to investigate reservoir impedance and thermal production. Because of the great depth of the geothermal reservoir, few data are available to characterize reservoir features and properties. As a result, the study focuses on the following three aspects: 1. model choice for predicting reservoir impedance and thermal breakthrough: after investigating three potential models (one single permeability model and two dual permeability models) for flow through fractured rock, it is decided to use single permeability model for further analysis; 2. well placement (horizontal vs. vertical) options: horizontal well placement seems to be more robust to heterogeneity and the impedance is more acceptable; 3. Prediction uncertainty: the most influential parameters are identified using a First-Order-Second-Moment uncertainty propagation analysis, and the uncertain range of the model predictions is estimated by performing a Monte Carlo simulation. Heterogeneity has a large impact on the perdition, therefore, is considered in the predictive model and uncertainty analysis. The numerical model results and uncertainty analysis are used for economic analysis. The dataset submitted here support the described study. Manuscript is submitted to Geothermics, will be linked once paper is accepted.

15 GEOTHERMAL ENERGY↗

Spatial Bayesian models project shifts in suitable habitat for Pacific Northwest tree species under climate change

Abstract We developed spatial Bayesian hierarchical models to assess potential climate change impacts on suitable habitat for five important tree species in the Pacific northwestern United States (California, Oregon, and Washington). Individual‐species models were fit with presence–absence data from forest inventory field plots and spatial relationships were specified through a conditional autoregressive model. This modeling approach allowed us to visualize uncertainty in response curves, map current and future prediction uncertainty, and provide interval estimates for change. Upward elevational or northward latitudinal shifts in climatically suitable habitat were projected for all species. Climate change impacts were the most damaging for noble fir ( Abies procera ), for which 79%–100% of the current range was projected to become climatically unsuitable by the 2080s. Although coastal Douglas‐fir ( Pseudotsuga menziesii var. menziesii ) has been projected by others to gain habitat in Canada, within our study area we projected a net loss of climatically suitable habitat (ca. 8000–31,400 km 2 ) under three of four future climate scenarios. A net loss in habitat was also projected for Oregon white oak ( Quercus garryana ) under three of four scenarios, with 40%–60% of the current range becoming unsuitable. Although there was no net loss of habitat for forest land blue oak under any scenario, other factors like competition may inhibit blue oak ( Quercus douglasii ) and white oak from occupying areas projected to increase in climatic suitability. Additionally, between 13% and 32% of blue oak's current range was projected to become unsuitable; some of these areas aligned with dieback following the 2012–2015 California drought, which our data set predates. Unlike the other four species, we projected a 17%–25% increase in climatically suitable habitat for California black oak ( Quercus kelloggii ), although 1%–20% of the current range was still projected to become unsuitable. Our findings indicate that, although some species will face more pressure in tracking climatically suitable habitat than others, climate change will impact the location of suitable habitat for many species.

Kralicek, Karin↗

MOOSE ProbML: Parallelizable Probabilistic Machine Learning and Uncertainty Quantification Capabilities

The Multiphysics Object Oriented Simulation Environment (MOOSE) is a widely used open- source finite element software for performing multiphysics multiscale simulations in a massively parallel fashion. Recently, the computational team at Idaho National Laboratory (INL) has implemented Probabilistic Machine Learning (ProbML) capabilities in MOOSE—in a parallelized fashion—and enable active learning with large-scale computational models for tasks such as surrogate model development, scale bridging, forward/inverse uncertainty quantification (UQ), Bayesian optimization, etc. This presentation summarizes these developments in MOOSE along with demonstrations on several real applications relevant to nuclear energy. At the fundamental level, samplers like Monte Carlo/Latin Hypercube, variance reduction, parallelized Markov Chain Monte Carlo (MCMC) support uncertainty propagation in both forward and inverse settings. These samplers can be integrated with the Gaussian processes (GP) suite in MOOSE, which offer several variants like scalar GPs, multi-output GPs, and deep GPs, to enable active learning. These GPs can be tuned using gradient-based optimization methods like Adam and its variants or gradient-free methods like the elliptical slice sampler (a variant of MCMC adept under Gaussian settings) for more complex covariance kernels or likelihoods whose gradient computations can be cumbersome. A variety of batch acquisition functions permit parallelized evaluation of the computational model and support different learning objectives with high efficiency like Bayesian inference, global surrogate development, optimization, etc. Furthermore, libtorch integration supports training, evaluation, and re-training of neural networks and other complex machine learning models in active learning settings. The impacts of these developments are shown on several real applications: (1) nuclear fuel inverse UQ and model inadequacy assessment using the Kennedy O’Hagan framework; (2) uncertainty aware surrogate modeling for additive manufacturing to predict field quantities; (3) nuclear reactor rare events analysis; and (4) complex fluid flow prediction using a global surrogate with quantified prediction uncertainty. Finally, the outlook of MOOSE ProbML is discussed for both outer-loop and inner-loop computations in the broad view to accelerate fuels and materials qualification, address gaps in knowledge and data, and assess new reactor/fuel systems.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Sensitivity Analysis of Irradiated Fueled Experiments using the MOOSE Framework [Slides]

Modeling and simulation (M&S) methods are able to predict uncertainties in experimental parameters (e.g., power and fission density) during irradiation. A shortfall exists in predicting how sensitive some of the parameters will behave during the experimental process. Sensitivity and Uncertainty Quantification (SUQ) is critical in support of qualification and licensing reactor fuels. The application of a method to quantify the uncertainty in these experiments is critical to the prediction of their performance. In this work, we propose the use of a polynomial chaos expansion (PCE) method to quantify the sensitive parameters in these simulations and, in an extension, their experimental surrogates. We propose to perform M&S using PCE uncertainty quantification on a previously irradiated fueled experiment in order to provide a validation case for Griffin and expand its use as a verification and validation (V&V) tool for experiments with a neutronics component. Griffin is an advanced, deterministic neutronics analysis code built using the MOOSE (multiphysics object-oriented simulation environment) framework which can provide state-of-the-art neutronic analysis on M&S of experiments. We will use the stochastic tools module (STM) in MOOSE to provide PCE uncertainty quantification on the proposed experimental setup. Idaho National Laboratory (INL) does not yet have an in-house developed code with V&V approval for experiments performed on-site; this work would provide a necessary addition of support for experiments performed at INL. The Nuclear Regulatory Commission (NRC) has explicitly requested uncertainties in calculated values such as fuel power and burnup, and the development of this capability would benefit the relationship between INL and the NRC.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Correlated Trajectory Uncertainty for Adaptive Sequential Decision Making

One of the great challenges with decision making tasks on real world systems is the fact that data is sparse and acquiring additional data is expensive. In these cases, it is often crucial to make a model of the environment to assist in making decisions. At the same time, limited data means that learned models are erroneous, making it just as important to equip the model with good predictive uncertainties. In the context of learning sequential decision making policies, these uncertainties can prove useful for informing which data to collect for the greatest improvement in policy performance \citep{mehta2021experimental, mehta2022exploration} or informing the policy about unsure regions of state and action space to avoid during test time \citep{yu2020mopo}. Additionally, assuming that realistic samples of the environment can be drawn, an adaptable policy can be trained that attempts to make optimal decisions for any given possible instance of the environment \citep{ghosh2022offline, chen2021offline}. In this work, we examine the so-called ``probabilistic neural network'' (PNN) model that is ubiquitous in model-based reinforcement learning (MBRL) works. We argue that while PNN models may have good marginal uncertainties, they form a distribution of non-smooth transition functions. Not only are these samples unrealistic and may hamper adaptability, but we also assert that this leads to poor uncertainty estimates when predicting multiple step trajectory estimates. To address this issue, we propose a simple sampling method that can be implemented on top of pre-existing models.We evaluate our sampling technique on a number of environments, including a realistic nuclear fusion task, and find that, not only do smooth transition function samples produce more calibrated uncertainties, but they also lead to better downstream performance for an adaptive policy.

Offline Reinforcement Learning↗

Distance preserving machine learning for uncertainty aware accelerator capacitance predictions

Abstract Accurate uncertainty estimations are essential for producing reliable machine learning models, especially in safety-critical applications such as accelerator systems. Gaussian process models are generally regarded as the gold standard for this task; however, they can struggle with large, high-dimensional datasets. Combining deep neural networks with Gaussian process approximation techniques has shown promising results, but dimensionality reduction through standard deep neural network layers is not guaranteed to maintain the distance information necessary for Gaussian process models. We build on previous work by comparing the use of the singular value decomposition against a spectral-normalized dense layer as a feature extractor for a deep neural Gaussian process approximation model and apply it to a capacitance prediction problem for the High Voltage Converter Modulators in the Oak Ridge Spallation Neutron Source. Our model shows improved distance preservation and predicts in-distribution capacitance values with less than 1% error.

43 PARTICLE ACCELERATORS↗

The North American Multi-Model Ensemble (NMME): Phase-1 Seasonal to Interannual Prediction, Phase-2 Toward Developing Intra-Seasonal Prediction

The recent US National Academies report "Assessment of Intraseasonal to Interannual Climate Prediction and Predictability" was unequivocal in recommending the need for the development of a North American Multi-Model Ensemble (NMME) operational predictive capability. Indeed, this effort is required to meet the specific tailored regional prediction and decision support needs of a large community of climate information users. The multi-model ensemble approach has proven extremely effective at quantifying prediction uncertainty due to uncertainty in model formulation, and has proven to produce better prediction quality (on average) then any single model ensemble. This multi-model approach is the basis for several international collaborative prediction research efforts, an operational European system and there are numerous examples of how this multi-model ensemble approach yields superior forecasts compared to any single model. Based on two NOAA Climate Test Bed (CTB) NMME workshops (February 18, and April 8, 2011) a collaborative and coordinated implementation strategy for a NMME prediction system has been developed and is currently delivering real-time seasonal-to-interannual predictions on the NOAA Climate Prediction Center (CPC) operational schedule. The hindcast and real-time prediction data is readily available (e.g., http://iridl.ldeo.columbia.edu/SOURCES/.Models/.NMME/) and in graphical format from CPC (http://origin.cpc.ncep.noaa.gov/products/people/wd51yf/NMME/index.html). Moreover, the NMME forecast are already currently being used as guidance for operational forecasters. This paper describes the new NMME effort, presents an overview of the multi-model forecast quality, and the complementary skill associated with individual models.

NOAA CPC↗

Sources of Uncertainty in the Prediction of LAI / fPAR from MODIS

To explicate the sources of uncertainty in the prediction of biophysical variables over space, consider the general equation: where z is a variable with values on some nominal, ordinal, interval or ratio scale; y is a vector of input variables; u is the spatial support of y and z ; x and u are the spatial locations of y and z , respectively; f is a model and B is the vector of the parameters of this model. Any y or z has a value and a spatial extent which is called its support. Viewed in this way, categories of uncertainty are from variable (e.g. measurement), parameter, positional. support and model (e.g. structural) sources. The prediction of Leaf Area Index (LAI) and the fraction of absorbed photosynthetically active radiation (fPAR) are examples of z variables predicted using model(s) as a function of y variables and spatially constant parameters. The MOD15 algorithm is an example of f, called f(sub 1), with parameters including those defined by one of six biome types and solar and view angles. The Leaf Canopy Model (LCM)2, a nested model that combines leaf radiative transfer with a full canopy reflectance model through the phase function, is a simpler though similar radiative transfer approach to f(sub 1). In a previous study, MOD15 and LCM2 gave similar results for the broadleaf forest biome. Differences between these two models can be used to consider the structural uncertainty in prediction results. In an effort to quantify each of the five sources of uncertainty and rank their relative importance for the LAI/fPAR prediction problem, we used recent data for an EOS Core Validation Site in the broadleaf biome with coincident surface reflectance, vegetation index, fPAR and LAI products from the Moderate Resolution Imaging Spectrometer (MODIS). Uncertainty due to support on the input reflectance variable was characterized using Landsat ETM+ data. Input uncertainties were propagated through the LCM2 model and compared with published uncertainties from the MOD15 algorithm.

Dungan, Jennifer L.↗

Low Activity Waste Glass Optimization with Property Models from Machine Learning, Part 2: Experimental Validation and Active Learning

The United States Department of Energy is responsible for managing legacy nuclear waste stored in underground tanks at the Hanford Site. To treat the waste, it is planned as the current baseline to separately vitrify low-activity waste (LAW) and high-level waste fractions. Previously, machine learning (ML) based glass property models (e.g., chemical durability, viscosity, electrical conductivity and SO3 solubility) were developed with prediction uncertainties. A waste glass optimization approach was then established to enable the capability of using these ML models in LAW glass formulation. In this study, the previous ML models were first experimentally validated, and the results were incorporated back into the database to update the ML models. The updated models and formulations showed increased waste loading while reducing the failure rate, demonstrating improved predictive accuracy, reduced uncertainties, and the effectiveness of active learning in guiding high-dimensional, nonlinear LAW glass design. This represents the first experimental validation of ML based LAW glass formulation, with practical benefits such as higher waste loading, shorter mission duration, and lower operational risk.

Lu, Xiaonan (ORCID:0000000179708148)↗

Multiclass classification experiments

We seek a multiclass classifier that can satisfy the following requirements. 1. Predict among at least three classes. 2. Handle a moderately large number of correlated predictors. 3. Handle mixed categorical and continuous predictors. 4. Handle missing values (in some way). 5. Be trained with about n = 1,000 responses, and 6. Quantify prediction uncertainty. Based on 5. above, it is assumed that accuracy of prediction is paramount (relative to, say, runtime). All methods considered in this document take on the order of minutes (most take seconds) on a 2.8 GHz Quad-Core Intel Core i7 processor with 16GB RAM. Secondly, quantification of uncertainty is assumed to be a secondary task, since, with this small data size, resampling methods (i.e. bootstrapping) are possible. Further studies should quantify the feasibility and accuracy of resampling methods, and/or other methods, for quantifying uncertainty. R was used for all software.

97 MATHEMATICS AND COMPUTING↗

An investigation on machine learning predictive accuracy improvement and uncertainty reduction using VAE-based data augmentation

The confluence of ultrafast computers with large memory, rapid progress in Machine Learning (ML) algorithms, and the availability of large datasets place multiple engineering fields at the threshold of dramatic progress. However, a unique challenge in nuclear engineering is data scarcity because experimentation on nuclear systems is usually more expensive and time-consuming than most other disciplines. One potential way to resolve the data scarcity issue is deep generative learning, which uses certain ML models to learn the underlying distribution of existing data and generate synthetic samples that resemble the real data. In this way, one can significantly expand the dataset to train more accurate predictive ML models. In this study, our objective is to evaluate the effectiveness of data augmentation using variational autoencoder (VAE)-based deep generative models. We investigated whether the data augmentation leads to improved accuracy in the predictions of a deep neural network (DNN) model trained using the augmented data. Additionally, the DNN prediction uncertainties are quantified using Bayesian Neural Networks (BNN) and conformal prediction (CP) to assess the impact on predictive uncertainty reduction. To test the proposed methodology, we used TRACE simulations of steady-state void fraction data based on the NUPEC Boiling Water Reactor Full-size Fine-mesh Bundle Test (BFBT) benchmark. Here, we found that augmenting the training dataset using VAEs has improved the DNN model’s predictive accuracy, improved the prediction confidence intervals, and reduced the prediction uncertainties.

Bayesian neural network↗

Parameter uncertainties for imperfect surrogate models in the low-noise regime

Abstract Bayesian regression determines model parameters by minimizing the expected loss, an upper bound to the true generalization error. However, this loss ignores model form error, or misspecification, meaning parameter uncertainties are significantly underestimated and vanish in the large data limit. As misspecification is the main source of uncertainty for surrogate models of low-noise calculations, such as those arising in atomistic simulation, predictive uncertainties are systematically underestimated. We analyze the true generalization error of misspecified, near-deterministic surrogate models, a regime of broad relevance in science and engineering. We show that posterior parameter distributions must cover every training point to avoid a divergence in the generalization error and design a compatible ansatz which incurs minimal overhead for linear models. The approach is demonstrated on model problems before application to thousand-dimensional datasets in atomistic machine learning. Our efficient misspecification-aware scheme gives accurate prediction and bounding of test errors in terms of parameter uncertainties, allowing this important source of uncertainty to be incorporated in multi-scale computational workflows.

Swinburne, Thomas D. (ORCID:0000000232554257)↗

Low-energy Electron-track Imaging for a Liquid Argon Time-projection-chamber Telescope Concept Using Probabilistic Deep Learning

The GammaTPC is an MeV-scale single-phase liquid argon time-projection-chamber gamma-ray telescope concept with a novel dual-scale pixel-based charge-readout system. It promises to enable a significant improvement in sensitivity to MeV-scale gamma rays over previous telescopes. The novel pixel-based charge readout allows for imaging of the tracks of electrons scattered by Compton interactions of incident gamma rays. The two primary contributors to the accuracy of a Compton telescope in reconstructing an incident gamma-ray’s original direction are its energy and position resolution. In this work, we focus on using deep learning to optimize the reconstruction of the initial position and direction of electrons scattered in Compton interactions, including using probabilistic models to estimate predictive uncertainty. We show that the deep-learning models are able to predict locations of Compton scatters of MeV-scale gamma rays from simulated 500 μm pixel-based data to better than 1 mm rms error and are sensitive to the initial direction of the scattered electron. We compare and contrast different deep-learning uncertainty estimation algorithms for reconstruction applications. Additionally, we show that event-by-event estimates of the uncertainty of the locations of the Compton scatters can be used to select those events that were reconstructed most accurately, leading to improvement in locating the origin of gamma-ray sources on the sky.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Employing Sensitivity Derivatives to Estimate Uncertainty Propagation in CFD

Two methods that exploit the availability of sensitivity derivatives are successfully employed to predict uncertainty propagation through Computational Fluid Dynamics (CFD) code for an inviscid airfoil problem. An approximate statistical second-moment method and a Sensitivity Derivative Enhanced Monte Carlo (SDEMC) method are successfully demonstrated on a two-dimensional problem. First- and second-order sensitivity derivatives of code output with respect to code input are obtained through an efficient incremental iterative approach. Given uncertainties in statistically independent, random, normally distributed flow parameters (input variables); these sensitivity derivatives enable one to formulate first- and second-order Taylor Series approximations for the mean and variance of CFD output quantities. Additionally, incorporation of the first-order sensitivity derivatives into the data reduction phase of a conventional Monte Carlo (MC) simulation allows for improved accuracy in determining the first moment of the CFD output. Both methods are compared to results generated using a conventional MC method. The methods that exploit the availability of sensitivity derivatives are found to be valid when considering small deviations from input mean values.

Putko, Michele M.↗

Regional-scale soil carbon predictions can be enhanced by transferring global-scale soil–environment relationships

Accurate modelling and mapping soil organic carbon are crucial for supporting soil health restoration and climate change mitigation at both regional and global scales. However, regional soil predictions often suffer from data scarcity and high prediction uncertainty. Utilizing a pre-trained global-to-regional soil carbon predictive model can be a potential solution to address this challenge. Despite its promise, how to construct and apply the global-scale model to enhance regional-scale soil carbon mapping remains largely unexplored. Here, we propose the Global Soil Carbon Pre-trained Model (GSoilCPM), a deep-learning-based domain adaptative model, to enhance regional-scale soil carbon predictions. Based on large amount of environmental covariate data and 106,167 soil samples across the globe, we verify our hypothesis of the effectiveness of this 'global-to-regional' modelling strategy. The pre-trained model can be then transferred and fine-tuned to bridge the regional- and global-scale soil–environment relationships. We applied and validated this modelling strategy in four regional-scale study areas, three in the Northern Hemisphere and one in the Southern Hemisphere, each with distinct environmental background. Compared to traditional modelling approaches as a baseline, four case studies all demonstrated significant improvement in prediction accuracy across diverse environments and varying data availabilities. The average percentage improvement across all regions is 10.93% (absolute values decreased by 1.20 g kg−1 averagely) in MAE and 29.04% (absolute values increased by 0.10 averagely) in CCC. The applicability and future horizons of using GSoilCPM were further discussed. We further reveal that regions with fewer soil samples or lower baseline accuracy benefit more from the pre-trained global model. Our findings highlight the advantages of leveraging the generalized knowledge from global models to enhance specifically localized soil modelling, positioning a potential paradigm shift in digital soil mapping, and far-reaching implications for soil monitoring and land management.

Deep learning↗

mphys-surrogate-model

This repository contains python scripts for building and studying reduced-order-modeling representations of droplet coalescence for eventual use in atmospheric models. The included data are generated from high-fidelity superdroplet methods and are utilized by machine learning pipelines to build data-driven models of droplet size distributions that evolve under coalescence. This repository further includes scripts to determine prediction (uncertainty) intervals on the data-driven model products based on conformal prediction.

Katona, JonasE [Lawrence Livermore National Labora↗