Improving Density Curve Predictions and Uncertainty Quantification with Functional Data Analysis [Slides]
Los Alamos National Laboratory (LANL) has developed advanced simulation methods for predicting subsurface flow.
97 MATHEMATICS AND COMPUTING↗
SEARCH · Engineering Papers
Results for “prediction uncertainty”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Cross-scale covariance for material property prediction
A simulation can stand its ground against an experiment only if its prediction uncertainty is known. The unknown accuracy of interatomic potentials (IPs) is a major source of prediction uncertainty, severely limiting the use of large-scale classical atomistic simulations in a wide range of scientific and engineering applications. Here we explore covariance between predictions of metal plasticity, from 178 large-scale (~10 8 atoms) molecular dynamics (MD) simulations, and a variety of indicator properties computed at small-scales (≤10 2 atoms). All simulations use the same 178 IPs. In a manner similar to statistical studies in public health, we analyze correlations of strength with indicators, identify the best predictor properties, and build a cross-scale “strength-on-predictors” regression model. This model is then used to estimate regression error over the statistical pool of IPs. Small-scale predictors found to be highly covariant with strength are computed using expensive quantum-accurate calculations and used to predict flow strength, within the statistical error bounds established in our study.
36 MATERIALS SCIENCE↗
Uncertainty Quantified Machine Learning for Street Level Flooding Predictions in Norfolk, Virginia
Everyday citizens, emergency responders, and critical infrastructure can be dramatically affected by street flooding caused by the increasing tidal levels and major storm events magnified by climate change. Low-level flooding, such as nuisance flooding, continues to increase in frequency, especially in cities like Norfolk, Virginia, which can expect nearly 200 flooding events by 2050 [1]. Recently, machine learning (ML) models have been leveraged to produce real-time predictions based on local weather and geographic conditions. However, ML models are known to produce unusual results when presented with data that varies from their training set. For decision-makers to determine the trustworthiness of the model's predictions, ML models need to quantify their prediction uncertainty. This study applies Deep Quantile Regression (DQR) to a previously published, LSTM-based model for hourly water depth predictions [2], and analyzes its out-of-distribution (OOD) performance.
Goldenberg, Steven↗
Time-temperature-superposition analysis of diverse datasets by the minimum-arclength method: long-term prediction with uncertainty margins
In a recent publication, we carried out an extensive analysis of an unsupervised method of determining optimum shift factors in time-temperature-superposition of accelerated-aging data that involves minimizing the vertical arclength to obtain the master curve. For synthetic Arrhenius data with a variety of noise distributions, the work showed that, in conjunction with bootstrap-resampling, the method can produce reliable estimates of the mean activation energy along with uncertainty quantification. Here, we apply the above method to six different datasets taken from the published literature and demonstrate accurate prediction of mean activation energy from the data as-is without the need for any pre-processing or fitting. We also compare uncertainty margins computed by second-order bootstrap with that by linear regression theory and show that the former appears to provide consistent margins in the presence of common noise types in real data, including intra-isotherm measurement-errors, sample-to-sample variations, and intrinsic deviation from perfect Arrhenius behavior.
36 MATERIALS SCIENCE↗
Precision calibration of calorimeter signals in the ATLAS experiment using an uncertainty-aware neural network
The ATLAS experiment at the Large Hadron Collider explores the use of modern neural networks for a multi-dimensional calibration of its calorimeter signal defined by clusters of topologically connected cells (topo-clusters). The Bayesian neural network (BNN) approach not only yields a continuous and smooth calibration function that improves performance relative to the standard calibration but also provides uncertainties on the calibrated energies for each topo-cluster. The results obtained by using a trained BNN are compared to the standard local hadronic calibration and to a calibration provided by training a deep neural network. The uncertainties predicted by the BNN are interpreted in the context of a fractional contribution to the systematic uncertainties of the trained calibration. They are also compared to uncertainty predictions obtained from an alternative estimator employing repulsive ensembles.
72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
Multi-Sensor Optimal Motion Planning for Radiological Contamination Surveys by Using Prediction-Difference Maps
Distributed and networked mobile sensor platforms using unmanned aerial and/or ground vehicles to survey areas of interest offer a safer and more efficient method for radiological contamination mapping; however, most applications rely on uniformly sweeping of the area in a raster-type motion without utilizing the information available in a dynamic sense. We have developed a fully autonomous optimal motion planning procedure for networks with two or more mobile sensors. The procedure utilizes well-established concepts of Gaussian processes in combination with control laws based on centroidal Voronoi tessellations to achieve optimal next-iteration sensor movements. A new method of informing optimal motion planning is proposed, whereby the absolute difference between the prior and current full-map prediction, referred to as the prediction-difference map, is used as the spatial density function within each Voronoi cell, providing immediate and iterative feedback for dynamic use of available information. The Gaussian process regression model used to estimate the contamination in unvisited locations also provides prediction uncertainties, and can be used as a quantitative metric to assess the confidence in the calculated contamination map; these estimates and prediction uncertainties are unavailable for standard uniform survey routines as they can only produce maps in the vicinity of observed locations. We present through simulation the achievable performance gains from using this new method by directly comparing to a uniform survey method. Results show that using the prediction-difference maps to inform motion planning procedures offers a faster rate of producing an accurate and convergent map relative to a uniform survey route.
47 OTHER INSTRUMENTATION↗
Gap-filling eddy covariance methane fluxes: Comparison of machine learning model predictions and uncertainties at FLUXNET-CH4 wetlands
Time series of methane fluxes measured by eddy-covariance require gap-filling to estimate annual emissions. Gap-filling methane fluxes is challenging because of high variability and complex responses to multiple drivers. To date, there is no widely established gap-filling standard for methane, with regards both to the best model algorithms and predictors. In this study, we address the need for standardization by synthesizing results of gap-filling methods applied at 17 wetland sites spanning boreal to tropical regions including all major wetlands classes and two rice paddies. We introduce new procedures for: 1) creating realistic artificial gap scenarios, 2) training and evaluating gap-filling models without overstating performance, and 3) predicting half-hourly methane fluxes and annual emissions with robust uncertainty estimates. We tested a conventional method (marginal distribution sampling) and four machine learning algorithms - penalized linear regression, artificial neural networks, random forests, and boosted decision trees - and four predictor sets, including temporal, meteorological, ecosystem carbon and energy flux, and soil predictors. We find that the conventional method can achieve similar median performance to the machine learning models but is worse than the best machine learning models and relatively insensitive to predictor choices. Of the machine learning models, decision tree algorithms performed the best in cross-validation experiments, even with a baseline predictor set, and artificial neural networks showed comparable performance when using all predictors. Soil temperature was frequently the most important predictor whilst water table depth was important at sites with substantial water table fluctuations, highlighting the value of data on soil conditions. Raw gap-filling uncertainties from the machine learning models were underestimated and we propose a method to calibrate uncertainties to observations. Finally, we gap-fill and provide summary evaluation metrics for all 81 sites in the FLUXNET-CH4 community dataset and publicly release the python code for model development, evaluation, and uncertainty estimation.
42 ENGINEERING↗
Model data for numerical evaluation of photosensitive tracers as a strategy for separating surface and subsurface transient storage in streams
This model-data archive pertains to a study aimed at the numerical evaluation of the photosensitive tracers as a potential strategy for separating the effects of surface and hyporheic storage zones (SSZs and HSZs, respectively). Separating effects of SSZ and HSZ are important for accurately representing stream function as HSZs and SSZs expose solutes to significantly different biogeochemical conditions. We perform numerical experiments using a multiscale reactive transport model for stream corridors implemented in ATS code, which allows for representing multiple storage zones with their respective travel time distributions and biogeochemistry. For each of the numerical experiment, we provide in this model-data archive python wrapper to drive ATS (forward_model.py), synthetic observation (synthetic_btc.py, BTC_observed.csv), forward model files including ATS input (multiscale_transport.tpl), PFLOTRAN inputs (reactions_channel.tpl, reactions_hz.tpl, reactions_sz.tpl), MCMC (mcmc_run.py), predictive uncertainty (pred_uncert.py, BTCs_simulated_day.csv, BTCs_simulated_night.csv), MCMC outputs (tracer_test_GR.npy, tracer_test_logps.npy, tracer_test_parameters.npy) and Jupyter notebook for post-processing and visualization (post-processing.ipynb). For the denitrification application, ATS input file (denitrification_multisubgrid.tpl), PFLOTRAN input files (denitrification_channel.in, denitrification_hz.in), predictive uncertainty (pred_uncert_denitrification.py, BTCs_simulated_DO.csv, BTCs_simulated_DOC.csv, BTCs_simulated_Nitrate.csv).
54 ENVIRONMENTAL SCIENCES↗
Parameter Uncertainty for Aircraft Aerodynamic Modeling using Recursive Least Squares
A real-time method was demonstrated for determining accurate uncertainty levels of stability and control derivatives estimated using recursive least squares and time-domain data. The method uses a recursive formulation of the residual autocorrelation to account for colored residuals, which are routinely encountered in aircraft parameter estimation and change the predicted uncertainties. Simulation data and flight test data for a subscale jet transport aircraft were used to demonstrate the approach. Results showed that the corrected uncertainties matched the observed scatter in the parameter estimates, and did so more accurately than conventional uncertainty estimates that assume white residuals. Only small differences were observed between batch estimates and recursive estimates at the end of the maneuver. It was also demonstrated that the autocorrelation could be reduced to a small number of lags to minimize computation and memory storage requirements without significantly degrading the accuracy of predicted uncertainty levels.
Grauer, Jared A.↗
Robust Vector BOTDA Signal Processing with Probabilistic Machine Learning
This paper presents a novel probabilistic machine learning (PML) framework to estimate the Brillouin frequency shift (BFS) from both Brillouin gain and phase spectra of a vector Brillouin optical time-domain analysis (VBOTDA). The PML framework is used to predict the Brillouin frequency shift (BFS) along the fiber and to assess its predictive uncertainty. We compare the predictions obtained from the proposed PML model with a conventional curve fitting method and evaluate the BFS uncertainty and data processing time for both methods. The proposed method is demonstrated using two BOTDA systems: (i) a BOTDA system with a 10 km sensing fiber and (ii) a vector BOTDA with a 25 km sensing fiber. The PML framework provides a pathway to enhance the VBOTDA system performance.
47 OTHER INSTRUMENTATION↗
DeepUQ: Assessing the Aleatoric Uncertainties from two Deep Learning Methods
Assessing the quality of aleatoric uncertainty estimates from uncertainty quantification (UQ) deep learning methods is important in scientific contexts, where uncertainty is physically meaningful and important to characterize and interpret exactly. We systematically compare aleatoric uncertainty measured by two UQ techniques, Deep Ensembles (DE) and Deep Evidential Regression (DER). Our method focuses on both zero-dimensional (0D) and two-dimensional (2D) data, to explore how the UQ methods function for different data dimensionalities. We investigate uncertainty injected on the input and output variables and include a method to propagate uncertainty in the case of input uncertainty so that we can compare the predicted aleatoric uncertainty to the known values. We experiment with three levels of noise. The aleatoric uncertainty predicted across all models and experiments scales with the injected noise level. However, the predicted uncertainty is miscalibrated to $\rm{std}(\sigma_{\rm al})$ with the true uncertainty for half of the DE experiments and almost all of the DER experiments. The predicted uncertainty is the least accurate for both UQ methods for the 2D input uncertainty experiment and the high-noise level. While these results do not apply to more complex data, they highlight that further research on post-facto calibration for these methods would be beneficial, particularly for high-noise and high-dimensional settings.
Nevin, Rebecca↗
Strategic and Tactical Functions in an Autonomous Air Traffic Management System
This paper evaluates, by means of fast-time simulation, performance of a candidate system for autonomous air traffic management. Advancing towards autonomy in air traffic management may be necessary in order for new air vehicle types such as electric Vertical Take Off and Landing (eVTOL) to operate safely and efficiently in airspace shared with conventional traffic. To account for uncertain prediction, autonomous air traffic management was divided into two integrated and coordinated subsystems: strategic scheduling, performed at predeparture, and tactical conflict detection and resolution, performed throughout the flight. The conflict detection and resolution subsystem contained a second tactical scheduling function that applied to flights operating in the airspace near the destination airport. This paper compares and contrasts the two subsystems and uses fast-time simulation to demonstrate the comparisons. A scenario of 54 flights inbound to Newark Liberty International Airport was simulated multiple times with different parameters. The scenario was created using flight plans recorded from the National Airspace System on a low weather, average traffic day in April 2018. Whereas the routes were not changed, the departure times of the flights were modified to increase arrival rates at the Newark runway and arrival meter fixes. Results of the simulations showed that the autonomous air traffic management system was able to safely manage the traffic, even with prediction uncertainty. In addition, they showed the importance of including flight holding maneuvers, in addition to path stretching, in conflict detection and resolution and of coordinating strategic and tactical scheduling. Finally, a tradeoff between absorbing the delay calculated by strategic scheduling on the ground versus in the air showed that taking most of the delay on the ground is cost effective for a simple idealized cost function. However, taking a little of the delay in the air prevented throughput on the runway from dropping for short periods due to trajectory prediction uncertainty.
air traffic control↗
Strategic and Tactical Functions in an Autonomous Air Traffic Management System
This paper evaluates, by means of fast-time simulation, performance of a candidate system for autonomous air traffic management. Advancing towards autonomy in air traffic management may be necessary in order for new air vehicle types such as electric Vertical Take Off and Landing (eVTOL) to operate safely and efficiently in airspace shared with conventional traffic. To account for uncertain prediction, autonomous air traffic management was divided into two integrated and coordinated subsystems: strategic scheduling, performed at predeparture, and tactical conflict detection and resolution, performed throughout the flight. The conflict detection and resolution subsystem contained a second tactical scheduling function that applied to flights operating in the airspace near the destination airport. This paper compares and contrasts the two subsystems and uses fast-time simulation to demonstrate the comparisons. A scenario of 54 flights inbound to Newark Liberty International Airport was simulated multiple times with different parameters. The scenario was created using flight plans recorded from the National Airspace System on a low weather, average traffic day in April 2018. Whereas the routes were not changed, the departure times of the flights were modified to increase arrival rates at the Newark runway and arrival meter fixes. Results of the simulations showed that the autonomous air traffic management system was able to safely manage the traffic, even with prediction uncertainty. In addition, they showed the importance of including flight holding maneuvers, in addition to path stretching, in conflict detection and resolution and of coordinating strategic and tactical scheduling. Finally, a tradeoff between absorbing the delay calculated by strategic scheduling on the ground versus in the air showed that taking most of the delay on the ground is cost effective for a simple idealized cost function. However, taking a little of the delay in the air prevented throughput on the runway from dropping for short periods due to trajectory prediction uncertainty.
air traffic control↗
A Bayesian Approach for In-Situ Stress Prediction and Uncertainty Quantification for Subsurface Engineering
Many subsurface engineering applications require accurate knowledge of the in-situ state of stress for their safe design and operation. Existing methods to meet this need primarily include field measurements for estimating one or more of the principal stresses from a borehole, or optimization methods for constructing a 3D geomechanical model in terms of geophysical measurements. These methods, however, often contain considerable uncertainty in estimating the state of stress. Here, in this paper, we build on a Bayesian approach to quantify uncertainty in stress estimations for subsurface engineering applications. This approach can provide an estimate of the 3D distribution of stress throughout the volume of interest and provide an estimate of the uncertainty arising from the stress measurement, the rheology parameters, and a paucity of measurements. The value of this approach is demonstrated using stress measurements from the In Salah carbon storage site, which was one of the world’s first industrial carbon capture and storage projects. This demonstration shows the application of this Bayesian approach for estimating the initial state of stress for In Salah and quantifying the uncertainty in the estimated stress. Also, an assessment of a maximum injection pressure to prevent geomechanical risks from CO 2 injection pressures is provided in terms of the probability distribution of the minimum principal stress quantified by the approach. With the In Salah case study, this paper demonstrates that using the Bayesian approach can provide additional insights for site explorations and/or project operations to make informed-site decisions for subsurface engineering applications.
58 GEOSCIENCES↗
Prediction and uncertainty quantification of shale well performance using multifidelity Monte Carlo
Uncertainty quantification is an integral component of reservoir management, especially considering the inherent uncertainty in subsurface systems. While a standard practice to estimate the uncertainty, Monte Carlo (MC) simulation is computationally intense when the sampling population comprises high-fidelity simulations. Alternatively, the Multi-fidelity Monte Carlo (MFMC) simulation overcomes this computational intensity by integrating low- and high-fidelity simulations. Our goal is to minimize the number of expensive high-fidelity simulations while maintaining accuracy and using numerous fast and cheap low-fidelity simulations to efficiently sample to input parameter space of interest. We selected gas production from unconventional wells to demonstrate the potential speedups and accuracy of the MFMC approach. The model fidelity usually determines the trade-off between accuracy and efficiency. While the high-fidelity model is more accurate, the low-fidelity model is more efficient. Our high-fidelity simulation includes reservoir simulations of a hydraulically fractured well. On the other hand, our low-fidelity model comprises the parallel-plate flow model. We used differential programming to efficiently solve the 1D flow model, where automatic differentiation is used to efficiently compute the gradients. We matched the production profile of high-fidelity simulations with our low-fidelity simulations. Then, we used a support vector regression to map the high- and low-fidelity input parameters. The mapping function is essential to tune the low-dimensional parameter space of the low-fidelity model to the high-dimensional parameter space of the high-fidelity model. We found that we can use a combination of 9 high fidelity and 10,000 low fidelity simulations to efficiently and accurately simulate pressure management. This method is at least two orders of magnitude faster than only using high-fidelity simulations. Finally, from a broader perspective, MFMC could efficiently estimate the uncertainty of various systems and models, integrating low- and high-fidelity models.
04 OIL SHALES AND TAR SANDS↗
Predictions and uncertainty estimates of reactor pressure vessel steel embrittlement using Machine learning
Not Available
Materials Science↗
Pairwise Difference Regression: A Machine Learning Meta-algorithm for Improved Prediction and Uncertainty Quantification in Chemical Search
Not Available
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Uncertainty-aware predictions of molecular x-ray absorption spectra using neural network ensembles
Not Available
97 MATHEMATICS AND COMPUTING↗