Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 685 records · Page 38
All sourcesNASA NTRSDOE OSTI
DOE OSTI2016

Modeling the Swift BAT trigger algorithm with machine learning

To draw inferences about gamma-ray burst (GRB) source populations based on Swift observations, it is essential to understand the detection efficiency of the Swift burst alert telescope (BAT). This study considers the problem of modeling the Swift/BAT triggering algorithm for long GRBs, a computationally expensive procedure, and models it using machine learning algorithms. A large sample of simulated GRBs from Lien et al. is used to train various models: random forests, boosted decision trees (with AdaBoost), support vector machines, and artificial neural networks. The best models have accuracies of ≳97% (≲3% error), which is a significant improvement on a cut in GRB flux, which has an accuracy of 89.6% (10.4% error). These models are then used to measure the detection efficiency of Swift as a function of redshift z, which is used to perform Bayesian parameter estimation on the GRB rate distribution. We find a local GRB rate density of n{sub 0}∼0.48{sub −0.23}{sup +0.41} Gpc{sup −3} yr{sup −1} with power-law indices of n{sub 1}∼1.7{sub −0.5}{sup +0.6} and n{sub 2}∼−5.9{sub −0.1}{sup +5.7} for GRBs above and below a break point of z{sub 1}∼6.8{sub −3.2}{sup +2.8}. This methodology is able to improve upon earlier studies by more accurately modeling Swift detection and using this for fully Bayesian model fitting.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

The Dark Energy Survey Year 3 high-redshift sample: selection, characterization, and analysis of galaxy clustering

ABSTRACT The fiducial cosmological analyses of imaging surveys like DES typically probe the Universe at redshifts z < 1. We present the selection and characterization of high-redshift galaxy samples using DES Year 3 data, and the analysis of their galaxy clustering measurements. In particular, we use galaxies that are fainter than those used in the previous DES Year 3 analyses and a Bayesian redshift scheme to define three tomographic bins with mean redshifts around z ∼ 0.9, 1.2, and 1.5, which extend the redshift coverage of the fiducial DES Year 3 analysis. These samples contain a total of about 9 million galaxies, and their galaxy density is more than 2 times higher than those in the DES Year 3 fiducial case. We characterize the redshift uncertainties of the samples, including the usage of various spectroscopic and high-quality redshift samples, and we develop a machine-learning method to correct for correlations between galaxy density and survey observing conditions. The analysis of galaxy clustering measurements, with a total signal to noise S/N ∼ 70 after scale cuts, yields robust cosmological constraints on a combination of the fraction of matter in the Universe Ωm and the Hubble parameter h, $\Omega _m h = 0.195^{+0.023}_{-0.018}$, and 2–3 per cent measurements of the amplitude of the galaxy clustering signals, probing galaxy bias and the amplitude of matter fluctuations, bσ8. A companion paper (in preparation) will present the cross-correlations of these high-z samples with cosmic microwave background lensing from Planck and South Pole Telescope, and the cosmological analysis of those measurements in combination with the galaxy clustering presented in this work.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Model-predictive optimal control of ferrofluidic microrobots in three-dimensional space

Ferrofluid microrobots have emerged as promising tools for minimally invasive medical procedures. Their unique properties to navigate complex fluids and reach otherwise inaccessible regions of the human body have enabled new applications in targeted drug delivery, tissue engineering, and diagnostics. Here, this paper proposes a model-predictive controller for the external magnetic manipulation of ferrofluid microrobots in three dimensions (3D). The internal optimization routine of the controller determines appropriate changes in the applied electromagnetic field to minimize the deviation between the actual and desired trajectories of the microrobot. A linear system governing locomotion is derived and used as the equality constraints of the optimization problems associated with the feedback index. In addition to ferrofluid droplets, the controller presented in this work may be applied to other magnetically-pulled microrobots. Several experiments are performed to validate the controller and showcase its ability to adapt to changes in system parameters such as the desired tracking trajectory and the size, orientation, deformation, and velocity of the microrobot. The accuracy of the controller is analyzed for each experiment, and the average error is found to be within 0.25 mm for small velocities. An additional experiment is performed to demonstrate significant improvement over a PID controller that is optimally tuned using Bayesian optimization. The results presented in this paper suggest that the proposed control algorithm could enable new microrobotic capabilities in minimally invasive medical procedures, lab-on-a-chip applications, and microfluidics.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2019

AUTOMATING SPENT FUEL DEFECT DETECTION WITH FUSED DCVD AND GET DATA

Within the safeguards regime, spent fuel monitoring has long been implemented to verify facility declarations. As the number and size of facilities under safeguards increase, so does time spent on highly repetitive spent fuel monitoring, making it an area ripe for automation. However, due to international proliferation concerns, automation must be robust and provide estimates of confidence in the results. Cerenkov Viewing Devices (CVDs) are widely implemented for spent fuel monitoring, but their single defect detection ability is limited. In this research, we augment CVD measurements with Gamma Emission Tomographer (GET) data in order to increase defect detection above the level of either detector method individually. Here we present the development and implementation of a Bayesian data fusion algorithm for classification of individual fuel rods as defects or nondefects across two burn-up and cooling time scenarios. Results show a nearly 75% single defect detection capability with a 10% false positive rate for 17x17 PWR fuel assemblies. The Bayesian framework also enables calculation of uncertainties associated with each classification. These results also show a moderate ability to generalize across both high burn-up and short cooling times as well as low burn-up and long cooling times.

Safeguards, image analysis, metric learning, Data ↗
DOE OSTI2024

Toward machine-learning-assisted PW-class high-repetition-rate experiments with solid targets

We present progress in utilizing a machine learning (ML) assisted optimization framework to study the trends in a parameter space defined by spectrally shaped, high-intensity, petawatt-class (8 J, 45 fs) laser pulses interacting with solid targets and give the first simulation-based overview of predicted trends. A neural network (NN) incorporating uncertainty quantification is trained to predict the number of hot electrons generated by the laser–target interaction as a function of pulse shaping parameters. The predictions of this NN serve as the basis function for a Bayesian optimization framework to navigate this space. For post-experimental evaluation, we compare two separate neural network (NN) models. One is based solely on data from experiments, and the other is trained only on ensemble particle-in-cell simulations. Reviewing the predicted and observed trends across the experiment-capable laser parameter search space, we find that both ML models predict a maximal increase in hot electron generation at a level of approximately 12%–18%; however, no statistically significant enhancement was observed in experiments. On direct comparison of the NN models, the average discrepancy is 8.5%, with a maximum of 30%. Since shot-to-shot fluctuations in experiments affect the observations, we evaluate the behavior of our optimization framework by performing virtual experiments that vary the number of repeated observations and the noise levels. Here, we discuss the implications of such a framework for future autonomous exploration platforms in high-repetition-rate experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Baycal

Bayesian Model Calibration (BayCal) toolkit is a software plugin for Risk Analysis Virtual Environment (RAVEN) framework, arming at inversely quantifying the uncertainties associated with simulation model parameters based on available experiment data. BayCal seeks statistical inference of the uncertain input parameters that are consistent with the available measurement data or observed data. The unique feature of BayCal is the capability to be linked with RAVEN to build corresponding calibration workflows for complex multi-physics simulations. In addition to be able to use the machine learning capability of RAVEN to significantly reduce the computational cost of expensive simulation models, another distinctive feature of BayCal is the capability to deal with high-dimensional correlated model outputs, such as time series observations at multiple locations, via principal component analysis (PCA) technique.

Wang, Congjian↗
DOE OSTI2025

An investigation on machine learning predictive accuracy improvement and uncertainty reduction using VAE-based data augmentation

The confluence of ultrafast computers with large memory, rapid progress in Machine Learning (ML) algorithms, and the availability of large datasets place multiple engineering fields at the threshold of dramatic progress. However, a unique challenge in nuclear engineering is data scarcity because experimentation on nuclear systems is usually more expensive and time-consuming than most other disciplines. One potential way to resolve the data scarcity issue is deep generative learning, which uses certain ML models to learn the underlying distribution of existing data and generate synthetic samples that resemble the real data. In this way, one can significantly expand the dataset to train more accurate predictive ML models. In this study, our objective is to evaluate the effectiveness of data augmentation using variational autoencoder (VAE)-based deep generative models. We investigated whether the data augmentation leads to improved accuracy in the predictions of a deep neural network (DNN) model trained using the augmented data. Additionally, the DNN prediction uncertainties are quantified using Bayesian Neural Networks (BNN) and conformal prediction (CP) to assess the impact on predictive uncertainty reduction. To test the proposed methodology, we used TRACE simulations of steady-state void fraction data based on the NUPEC Boiling Water Reactor Full-size Fine-mesh Bundle Test (BFBT) benchmark. Here, we found that augmenting the training dataset using VAEs has improved the DNN model’s predictive accuracy, improved the prediction confidence intervals, and reduced the prediction uncertainties.

Bayesian neural network↗
DOE OSTI2021

Training Spiking Neural Networks with Synaptic Plasticity under Integer Representation

Neuromorphic computing is emerging as a promising Beyond Moore computing paradigm that employs event-triggered computation and non-von Neumann hardware. Spike Timing Dependent Plasticity (STDP) is a well-known bio-inspired learning rule that relies on activities of locally connected neurons to adjust the weights of their respective synapses. In this work, we analyze a basic STDP rule and its sensitivity on the different hyperparameters for training spiking neural networks (SNNs) with supervision, customized for a neuromorphic hardware implementation with integer weights. We compare the classification performance on four UCI datasets (iris, wine, breast cancer and digits) that depict varying levels of complexity. We perform a search for optimal set of hyperparameters using both grid search and Bayesian optimization. Through the use of Bayesian optimization, we show the general trends in hyperparameter sensitivity in SNN classification problem. With the best sets of hyperparameters, we achieve accuracies comparable to some of the best performing SNNs on these four datasets. With a highly optimized supervised STDP rule we show that these accuracies can be achieved with just 20 epochs of training.

Kulkarni, Shruti↗
DOE OSTI2025

Efficient and Selective Chemical Transformations in Highly Charged and Confined Nanodroplets

The acceleration of chemical reaction rates and the increased product selectivity in microdroplets compared to that in bulk solutions has become a topic of increasing interest that has been extensively characterized by electrospray ionization mass spectrometry (ESI-MS). However, the sources of this acceleration and the detailed relationships between droplet properties and resulting reaction rate acceleration are still under debate. Moreover, droplet properties are governed by multiple interrelated experimental parameters, i.e., electrospray voltage, solution flow rate, etc., which makes it difficult and time-consuming to explore this diverse parameter space using traditional manual experimental or computational approaches. In this work, we developed an automated experimental platform integrating reactions in controlled charged microdroplet environments with ESI-MS characterization and sequential hybrid Bayesian modeling, as well as an optimal experimental design framework, to achieve multidimensional parameter optimization for higher reaction turnover rates, based on a model reaction of tetraethylenepentamine (TEPA) with carbon dioxide. With the current platform, we have achieved automated scans with a range of electrospray voltages and solution flow rates, and determined and optimized parameter settings to achieve increased reaction turnovers. We have also linked this platform to the underlying properties of droplets via a hybrid model incorporating physics, high-level theoretical calculations, and machine learning (ML) approaches. The autonomous platform is broadly applicable to a range of chemical reactions relevant to DOE’s mission in chemical separations, catalysis, and materials synthesis.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Generalized Bayesian MARS: Tools for Stochastic Computer Model Emulation

The multivariate adaptive regression spline (MARS) approach of Friedman and its Bayesian counterpart are effective approaches for the emulation of computer models. The traditional assumption of Gaussian errors limits the usefulness of MARS, and many popular alternatives, when dealing with stochastic computer models. Here, we propose a generalized Bayesian MARS (GBMARS) framework which admits the broad class of generalized hyperbolic distributions as the induced likelihood function. This allows us to develop tools for the emulation of stochastic simulators which are parsimonious, scalable, and interpretable and require minimal tuning, while providing powerful predictive and uncertainty quantification capabilities. GBMARS is capable of robust regression with t distributions, quantile regression with asymmetric Laplace distributions, and a general form of “Normal-Wald” regression in which the shape of the error distribution and the structure of the mean function are learned simultaneously. We demonstrate the effectiveness of GBMARS on various stochastic computer models, and we show that it compares favorably to several popular alternatives.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

A robust Bayesian approach to modeling epistemic uncertainty in common-cause failure models

In a standard Bayesian approach to the alpha-factor model for common-cause failure, a precise Dirichlet prior distribution models epistemic uncertainty in the alpha-factors. This Dirichlet prior is then updated with observed data to obtain a posterior distribution, which forms the basis for further inferences. In this paper, we adapt the imprecise Dirichlet model of Walley to represent epistemic uncertainty in the alpha-factors. In this approach, epistemic uncertainty is expressed more cautiously via lower and upper expectations for each alpha-factor, along with a learning parameter which determines how quickly the model learns from observed data. For this application, we focus on elicitation of the learning parameter, and find that values in the range of 1 to 10 seem reasonable. The approach is compared with Kelly and Atwood's minimally informative Dirichlet prior for the alpha-factor model, which incorporated precise mean values for the alpha-factors, but which was otherwise quite diffuse. Next, we explore the use of a set of Gamma priors to model epistemic uncertainty in the marginal failure rate, expressed via a lower and upper expectation for this rate, again along with a learning parameter. As zero counts are generally less of an issue here, we find that the choice of this learning parameter is less crucial. Finally, we demonstrate how both epistemic uncertainty models can be combined to arrive at lower and upper expectations for all common-cause failure rates. Thereby, we effectively provide a full sensitivity analysis of common-cause failure rates, properly reflecting epistemic uncertainty of the analyst on all levels of the common-cause failure model.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2024

Forward variable selection enables fast and accurate dynamic system identification with Karhunen-Loève decomposed Gaussian processes

A promising approach for scalable Gaussian processes (GPs) is the Karhunen-Loève (KL) decomposition, in which the GP kernel is represented by a set of basis functions which are the eigenfunctions of the kernel operator. Such decomposed kernels have the potential to be very fast, and do not depend on the selection of a reduced set of inducing points. However KL decompositions lead to high dimensionality, and variable selection thus becomes paramount. This paper reports a new method of forward variable selection, enabled by the ordered nature of the basis functions in the KL expansion of the Bayesian Smoothing Spline ANOVA kernel (BSS-ANOVA), coupled with fast Gibbs sampling in a fully Bayesian approach. It quickly and effectively limits the number of terms, yielding a method with competitive accuracies, training and inference times for tabular datasets of low feature set dimensionality. Theoretical computational complexities are O ( N P 2 ) in training and O ( P ) per point in inference, where N is the number of instances and P the number of expansion terms. The inference speed and accuracy makes the method especially useful for dynamic systems identification, by modeling the dynamics in the tangent space as a static problem, then integrating the learned dynamics using a high-order scheme. The methods are demonstrated on two dynamic datasets: a ‘Susceptible, Infected, Recovered’ (SIR) toy problem, along with the experimental ‘Cascaded Tanks’ benchmark dataset. Comparisons on the static prediction of time derivatives are made with a random forest (RF), a residual neural network (ResNet), and the Orthogonal Additive Kernel (OAK) inducing points scalable GP, while for the timeseries prediction comparisons are made with LSTM and GRU recurrent neural networks (RNNs) along with the SINDy package.

Hayes, Kyle↗
DOE OSTI2020

An active learning high-throughput microstructure calibration framework for solving inverse structure–process problems in materials informatics

Determining a process–structure–property relationship is the holy grail of materials science, where both computational prediction in the forward direction and materials design in the inverse direction are essential. Problems in materials design are often considered in the context of process–property linkage by bypassing the materials structure, or in the context of structure–property linkage as in microstructure-sensitive design problems. However, there is a lack of research effort in studying materials design problems in the context of process–structure linkage, which has a great implication in reverse engineering. In this paper, given a target microstructure, we propose an active learning high-throughput microstructure calibration framework to derive a set of processing parameters, which can produce an optimal microstructure that is statistically equivalent to the target microstructure. The proposed framework is formulated as a noisy multi-objective optimization problem, where each objective function measures a deterministic or statistical difference of the same microstructure descriptor between a candidate microstructure and a target microstructure. Furthermore, to significantly reduce the physical waiting wall-time, we enable the high-throughput feature of the microstructure calibration framework by adopting an asynchronously parallel Bayesian optimization to exploit high-performance computing resources. Case studies in additive manufacturing and grain growth are used to demonstrate the applicability of the proposed framework, where kinetic Monte Carlo (kMC) simulation is used as a forward predictive model, such that for a given target microstructure, the target processing parameters that produced this microstructure are successfully recovered.

36 MATERIALS SCIENCE↗
DOE OSTI2023

Fusion of Multiple Models for Improving Gross Primary Production Estimation With Eddy Covariance Data Based on Machine Learning

Abstract Terrestrial gross primary production (GPP) represents the magnitude of CO 2 uptake through vegetation photosynthesis, and is a key variable for carbon cycles between the biosphere and atmosphere. Light use efficiency (LUE) models have been widely used to estimate GPP for its physiological mechanisms and availability of data acquisition and implementation, yet each individual GPP model has exhibited large uncertainties due to input errors and model structure, and further studies of systematic validation, comparison, and fusion of those models with eddy covariance (EC) site data across diverse ecosystem types are still needed in order to further improve GPP estimation. We here compared and fused five GPP models (VPM, EC‐LUE, GOL‐PEM, CHJ, and C‐Fix) across eight ecosystems based on FLUXNET2015 data set using the ensemble methods of Bayesian Model Averaging (BMA), Support Vector Machine (SVM), and Random Forest (RF) separately. Our results showed that for individual models, EC‐LUE gave a better performance to capture interannual variability of GPP than other models, followed by VPM and GLO‐PEM, while CHJ and C‐Fix were more limited in their estimation performance. We found RF and SVM were superior to BMA on merging individual models at various plant functional types (PFTs) and at the scale of individual sites. On the basis of individual models, the fusion methods of BMA, SVM, and RF were examined by a five‐fold cross validation for each ecosystem type, and each method successfully improved the average accuracy of estimation by 8%, 18%, and 19%, respectively.

Environmental Sciences & Ecology↗
DOE OSTI2023

DeepONet-grid-UQ: A trustworthy deep operator framework for predicting the power grid’s post-fault trajectories

This paper proposes a novel data-driven method for the reliable prediction of the power grid’s post-fault trajectories, i.e., the power grid’s dynamic response after a disturbance or fault. Here, the proposed method is based on the recently proposed concept of Deep Operator Networks (DeepONets). Unlike traditional neural networks that learn to approximate functions, DeepONets are designed to approximate nonlinear operators, i.e., mappings between infinite-dimensional spaces. Under this operator framework, we design a novel and efficient DeepONet that (i) takes as inputs the trajectories collected before and during the fault and (ii) outputs the predicted post-fault trajectories. In addition, we endow our method with the much-needed ability to balance efficiency with reliable/trustworthy predictions via uncertainty quantification. To this end, we propose and compare two novel methods that enable quantifying the predictive uncertainty. First, we propose a Bayesian DeepONet (B-DeepONet) that uses stochastic gradient Hamiltonian Monte-Carlo to sample from the posterior distribution of the DeepONet trainable parameters. Then, we design a Probabilistic DeepONet (Prob-DeepONet) that uses a probabilistic training strategy to enable quantifying uncertainty at virtually no extra computational cost. Finally, we validate the proposed methods’ predictive power and uncertainty quantification capability using the New York-New England power grid model.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

On the Need to Align Intent and Implementation in Uncertainty Quantification for Machine Learning

Quantifying uncertainties for machine learning (ML) models is a foundational challenge in modern data analysis. This challenge is compounded by at least two key aspects of the field: (a) inconsistent terminology surrounding uncertainty and estimation across disciplines, and (b) the varying technical requirements for establishing trustworthy uncertainties in diverse problem contexts. In this position paper, we aim to clarify the depth of these challenges by identifying these inconsistencies and articulating how different contexts impose distinct epistemic demands. We examine the current landscape of estimation targets (e.g., prediction, inference, simulation-based inference), uncertainty constructs (e.g., frequentist, Bayesian, fiducial), and the approaches used to map between them. Drawing on the literature, we highlight and explain examples of problematic mappings. To help address these issues, we advocate for standards that promote alignment between the \textit{intent} and \textit{implementation} of uncertainty quantification (UQ) approaches. We discuss several axes of trustworthiness that are necessary (if not sufficient) for reliable UQ in ML models, and show how these axes can inform the design and evaluation of uncertainty-aware ML systems. Our practical recommendations focus on scientific ML, offering illustrative cases and use scenarios, particularly in the context of simulation-based inference (SBI).

Trivedi, Shubhendu [MIT] (ORCID:0000000312374301)↗
DOE OSTI2019

Crowdsourced reliable labeling of safety-rule violations on images of complex construction scenes for advanced vision-based workplace safety

Construction workplace hazard detection requires engineers to analyze scenes manually against many safety rules, which is time-consuming, labor-intensive, and error-prone. Computer vision algorithms are yet to achieve reliable discrimination of anomalous and benign object relations underpinning safety violation detections. Recently developed deep learning-based computer vision algorithms need tens of thousands of images, including labels of the safety rules violated, in order to train deep-learning networks for acquiring spatiotemporal reasoning capacity in complex workplaces. Such training processes need human experts to label images and indicate whether the relationship between the worker, resource, and equipment in the scenes violate spatiotemporal arrangement rules for safe and productive operations. False alarms in those manual labels (labeling no-violation images as having violations) can significantly mislead the machine learning process and result in computer vision models that produce inaccurate hazard detections. Compared with false alarms, another type of mislabels, false negatives (labeling images having violations as “no violations”), seem to have fewer impacts on the reliability of the trained computer vision models.

97 MATHEMATICS AND COMPUTING↗