Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 757 records · Page 42
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

A Bayesian Approach for Estimating Uncertainty in Stochastic Economic Dispatch considering Wind Power Penetration

The increasing penetration of renewable energy resources in power systems, represented as random processes, converts the traditional deterministic economic dispatch problem into a stochastic one. To estimate the uncertainty in this stochastic economic dispatch problem for forecasting purposes, the conventional Monte-Carlo method is prohibitively time-consuming for practical applications. To overcome this problem, here we propose a novel Gaussian-process-emulator-based approach to quantify the uncertainty in the stochastic economic dispatch considering wind power penetration. Facing high-dimensional real-world data representing the correlated uncertainties from wind generation, a manifold-learning-based Isomap algorithm is proposed to efficiently represent the low-dimensional hidden probabilistic structure of the data. In this low-dimensional latent space, with Latin hypercube sampling as the computer experimental design, a Gaussian-process emulator is used, for the first time, to serve as a nonparametric, surrogate model for the original complicated stochastic economic dispatch model. This reduced-order representative allows us to evaluate the economic dispatch solver at sampled values with a negligible computational cost while maintaining a desirable accuracy. Simulation results conducted on the IEEE 118-bus test system reveal the impressive performance of the proposed method.

17 WIND ENERGY↗
DOE OSTI2026

Characterization and automated optimization of laser-driven proton beams from converging liquid sheet jet targets

Compact, stable, and versatile laser-driven ion sources hold great promise for applications ranging from medicine to materials science and fundamental physics. While single-shot sources have demonstrated favorable beam properties, including the peak fluxes necessary for several applications, high-repetition-rate operation will be necessary to generate and sustain the high average flux needed for many of the most exciting applications of laser-driven ion sources. Further, to navigate through the high-dimensional space of laser and target parameters toward experimental optima, it is essential to develop ion acceleration platforms compatible with machine learning techniques and capable of autonomous real-time optimization. Here, we present a multi-Hz ion acceleration platform employing a liquid sheet jet target. We characterize the laser-plasma interaction and the laser-driven proton beam across a variety of key parameters governing the interaction using an extensive suite of online diagnostics. We also demonstrate real-time, closed-loop optimization of the ion beam maximum energy by tuning the laser wave front using a Bayesian optimization scheme. This approach increased the maximum proton energy by 11% compared to a manually optimized wave front by enhancing the energy concentration within the laser focal spot, demonstrating the potential for closed-loop optimization schemes to tune future ion accelerators for robust high-repetition-rate operation.

Glenn, G. D. [SLAC National Accelerator Laboratory↗
NASA NTRS2004

Lattice Duality: The Origin of Probability and Entropy

Bayesian probability theory is an inference calculus, which originates from a generalization of inclusion on the Boolean lattice of logical assertions to a degree of inclusion represented by a real number. Dual to this lattice is the distributive lattice of questions constructed from the ordered set of down-sets of assertions, which forms the foundation of the calculus of inquiry-a generalization of information theory. In this paper we introduce this novel perspective on these spaces in which machine learning is performed and discuss the relationship between these results and several proposed generalizations of information theory in the literature.

Knuth, Kevin H.↗
DOE OSTI2016

WE-AB-207B-02: A Bayesian Network Approach for Joint Prediction of Tumor Control and Radiation Pneumonitis (RP) in Non-Small-Cell Lung Cancer (NSCLC)

Purpose: NSCLC radiotherapy treatment is a trade-off between controlling the tumor while limiting radiation-induced toxicities. Here we identify hierarchical biophysical relationships that could simultaneously influence both local control (LC) and RP by using an integrated Bayesian Networks (BN) approach. Methods: We studied 79 NSCLC patients treated on prospective protocol with 56 cases of LC and 21 events of RP. Beyond dosimetric information, each patient had 193 features including 12 clinical factors, 60 circulating blood cytokines before and during radiotherapy, 62 microRNAs, and 59 single-nucleotide polymorphisms (SNPs). The most relevant biophysical predictors for both LC and RP were identified using a Markov blanket local discovery algorithm and the corresponding BN was constructed using a score-learning algorithm. The area under the free-response receiver operating characteristics (AU-FROC) was used for performance evaluation. Cross-validation was employed to guard against overfitting pitfalls. Results: A BN revealing the biophysical interrelationships jointly in terms of LC and RP was developed and evaluated. The integrated BN included two SNPs, one microRNA, one clinical factor, three pre-treatment cytokines, relative changes of two cytokines between pre and during-treatment, and gEUDs of the GTV (a=-20) and lung (a=1). On cross-validation, the AUC prediction of independent LC was 0.85 (95% CI: 0.75–0.95) and RP was 0.83 (0.73–0.92). The AU-FROC of the integrated BN to predict both LC/RP was 0.81 (0.71–0.90) based on 2000 stratified bootstrap, indicating minimal loss in joint prediction power. Conclusions: We developed a new approach for multiple outcome utility application in radiotherapy based on integrated BN techniques. The BN developed from large-scale retrospective data is able to simultaneously predict LC and RP in NSCLC treatments based on individual patient characteristics. The joint prediction is only slightly compromised compared to independent predictions. Our approach shows promise for use in clinical decision support system for personalized radiotherapy subject to multiple endpoints. These studies were supported by a grant from the NCI/NIH P01-CA59827.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2016

Modeling the Swift BAT trigger algorithm with machine learning

To draw inferences about gamma-ray burst (GRB) source populations based on Swift observations, it is essential to understand the detection efficiency of the Swift burst alert telescope (BAT). This study considers the problem of modeling the Swift/BAT triggering algorithm for long GRBs, a computationally expensive procedure, and models it using machine learning algorithms. A large sample of simulated GRBs from Lien et al. is used to train various models: random forests, boosted decision trees (with AdaBoost), support vector machines, and artificial neural networks. The best models have accuracies of ≳97% (≲3% error), which is a significant improvement on a cut in GRB flux, which has an accuracy of 89.6% (10.4% error). These models are then used to measure the detection efficiency of Swift as a function of redshift z, which is used to perform Bayesian parameter estimation on the GRB rate distribution. We find a local GRB rate density of n{sub 0}∼0.48{sub −0.23}{sup +0.41} Gpc{sup −3} yr{sup −1} with power-law indices of n{sub 1}∼1.7{sub −0.5}{sup +0.6} and n{sub 2}∼−5.9{sub −0.1}{sup +5.7} for GRBs above and below a break point of z{sub 1}∼6.8{sub −3.2}{sup +2.8}. This methodology is able to improve upon earlier studies by more accurately modeling Swift detection and using this for fully Bayesian model fitting.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

The Dark Energy Survey Year 3 high-redshift sample: selection, characterization, and analysis of galaxy clustering

ABSTRACT The fiducial cosmological analyses of imaging surveys like DES typically probe the Universe at redshifts z < 1. We present the selection and characterization of high-redshift galaxy samples using DES Year 3 data, and the analysis of their galaxy clustering measurements. In particular, we use galaxies that are fainter than those used in the previous DES Year 3 analyses and a Bayesian redshift scheme to define three tomographic bins with mean redshifts around z ∼ 0.9, 1.2, and 1.5, which extend the redshift coverage of the fiducial DES Year 3 analysis. These samples contain a total of about 9 million galaxies, and their galaxy density is more than 2 times higher than those in the DES Year 3 fiducial case. We characterize the redshift uncertainties of the samples, including the usage of various spectroscopic and high-quality redshift samples, and we develop a machine-learning method to correct for correlations between galaxy density and survey observing conditions. The analysis of galaxy clustering measurements, with a total signal to noise S/N ∼ 70 after scale cuts, yields robust cosmological constraints on a combination of the fraction of matter in the Universe Ωm and the Hubble parameter h, $\Omega _m h = 0.195^{+0.023}_{-0.018}$, and 2–3 per cent measurements of the amplitude of the galaxy clustering signals, probing galaxy bias and the amplitude of matter fluctuations, bσ8. A companion paper (in preparation) will present the cross-correlations of these high-z samples with cosmic microwave background lensing from Planck and South Pole Telescope, and the cosmological analysis of those measurements in combination with the galaxy clustering presented in this work.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Model-predictive optimal control of ferrofluidic microrobots in three-dimensional space

Ferrofluid microrobots have emerged as promising tools for minimally invasive medical procedures. Their unique properties to navigate complex fluids and reach otherwise inaccessible regions of the human body have enabled new applications in targeted drug delivery, tissue engineering, and diagnostics. Here, this paper proposes a model-predictive controller for the external magnetic manipulation of ferrofluid microrobots in three dimensions (3D). The internal optimization routine of the controller determines appropriate changes in the applied electromagnetic field to minimize the deviation between the actual and desired trajectories of the microrobot. A linear system governing locomotion is derived and used as the equality constraints of the optimization problems associated with the feedback index. In addition to ferrofluid droplets, the controller presented in this work may be applied to other magnetically-pulled microrobots. Several experiments are performed to validate the controller and showcase its ability to adapt to changes in system parameters such as the desired tracking trajectory and the size, orientation, deformation, and velocity of the microrobot. The accuracy of the controller is analyzed for each experiment, and the average error is found to be within 0.25 mm for small velocities. An additional experiment is performed to demonstrate significant improvement over a PID controller that is optimally tuned using Bayesian optimization. The results presented in this paper suggest that the proposed control algorithm could enable new microrobotic capabilities in minimally invasive medical procedures, lab-on-a-chip applications, and microfluidics.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2019

AUTOMATING SPENT FUEL DEFECT DETECTION WITH FUSED DCVD AND GET DATA

Within the safeguards regime, spent fuel monitoring has long been implemented to verify facility declarations. As the number and size of facilities under safeguards increase, so does time spent on highly repetitive spent fuel monitoring, making it an area ripe for automation. However, due to international proliferation concerns, automation must be robust and provide estimates of confidence in the results. Cerenkov Viewing Devices (CVDs) are widely implemented for spent fuel monitoring, but their single defect detection ability is limited. In this research, we augment CVD measurements with Gamma Emission Tomographer (GET) data in order to increase defect detection above the level of either detector method individually. Here we present the development and implementation of a Bayesian data fusion algorithm for classification of individual fuel rods as defects or nondefects across two burn-up and cooling time scenarios. Results show a nearly 75% single defect detection capability with a 10% false positive rate for 17x17 PWR fuel assemblies. The Bayesian framework also enables calculation of uncertainties associated with each classification. These results also show a moderate ability to generalize across both high burn-up and short cooling times as well as low burn-up and long cooling times.

Safeguards, image analysis, metric learning, Data ↗
DOE OSTI2024

Toward machine-learning-assisted PW-class high-repetition-rate experiments with solid targets

We present progress in utilizing a machine learning (ML) assisted optimization framework to study the trends in a parameter space defined by spectrally shaped, high-intensity, petawatt-class (8 J, 45 fs) laser pulses interacting with solid targets and give the first simulation-based overview of predicted trends. A neural network (NN) incorporating uncertainty quantification is trained to predict the number of hot electrons generated by the laser–target interaction as a function of pulse shaping parameters. The predictions of this NN serve as the basis function for a Bayesian optimization framework to navigate this space. For post-experimental evaluation, we compare two separate neural network (NN) models. One is based solely on data from experiments, and the other is trained only on ensemble particle-in-cell simulations. Reviewing the predicted and observed trends across the experiment-capable laser parameter search space, we find that both ML models predict a maximal increase in hot electron generation at a level of approximately 12%–18%; however, no statistically significant enhancement was observed in experiments. On direct comparison of the NN models, the average discrepancy is 8.5%, with a maximum of 30%. Since shot-to-shot fluctuations in experiments affect the observations, we evaluate the behavior of our optimization framework by performing virtual experiments that vary the number of repeated observations and the noise levels. Here, we discuss the implications of such a framework for future autonomous exploration platforms in high-repetition-rate experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Baycal

Bayesian Model Calibration (BayCal) toolkit is a software plugin for Risk Analysis Virtual Environment (RAVEN) framework, arming at inversely quantifying the uncertainties associated with simulation model parameters based on available experiment data. BayCal seeks statistical inference of the uncertain input parameters that are consistent with the available measurement data or observed data. The unique feature of BayCal is the capability to be linked with RAVEN to build corresponding calibration workflows for complex multi-physics simulations. In addition to be able to use the machine learning capability of RAVEN to significantly reduce the computational cost of expensive simulation models, another distinctive feature of BayCal is the capability to deal with high-dimensional correlated model outputs, such as time series observations at multiple locations, via principal component analysis (PCA) technique.

Wang, Congjian↗
DOE OSTI2025

An investigation on machine learning predictive accuracy improvement and uncertainty reduction using VAE-based data augmentation

The confluence of ultrafast computers with large memory, rapid progress in Machine Learning (ML) algorithms, and the availability of large datasets place multiple engineering fields at the threshold of dramatic progress. However, a unique challenge in nuclear engineering is data scarcity because experimentation on nuclear systems is usually more expensive and time-consuming than most other disciplines. One potential way to resolve the data scarcity issue is deep generative learning, which uses certain ML models to learn the underlying distribution of existing data and generate synthetic samples that resemble the real data. In this way, one can significantly expand the dataset to train more accurate predictive ML models. In this study, our objective is to evaluate the effectiveness of data augmentation using variational autoencoder (VAE)-based deep generative models. We investigated whether the data augmentation leads to improved accuracy in the predictions of a deep neural network (DNN) model trained using the augmented data. Additionally, the DNN prediction uncertainties are quantified using Bayesian Neural Networks (BNN) and conformal prediction (CP) to assess the impact on predictive uncertainty reduction. To test the proposed methodology, we used TRACE simulations of steady-state void fraction data based on the NUPEC Boiling Water Reactor Full-size Fine-mesh Bundle Test (BFBT) benchmark. Here, we found that augmenting the training dataset using VAEs has improved the DNN model’s predictive accuracy, improved the prediction confidence intervals, and reduced the prediction uncertainties.

Bayesian neural network↗
DOE OSTI2021

Training Spiking Neural Networks with Synaptic Plasticity under Integer Representation

Neuromorphic computing is emerging as a promising Beyond Moore computing paradigm that employs event-triggered computation and non-von Neumann hardware. Spike Timing Dependent Plasticity (STDP) is a well-known bio-inspired learning rule that relies on activities of locally connected neurons to adjust the weights of their respective synapses. In this work, we analyze a basic STDP rule and its sensitivity on the different hyperparameters for training spiking neural networks (SNNs) with supervision, customized for a neuromorphic hardware implementation with integer weights. We compare the classification performance on four UCI datasets (iris, wine, breast cancer and digits) that depict varying levels of complexity. We perform a search for optimal set of hyperparameters using both grid search and Bayesian optimization. Through the use of Bayesian optimization, we show the general trends in hyperparameter sensitivity in SNN classification problem. With the best sets of hyperparameters, we achieve accuracies comparable to some of the best performing SNNs on these four datasets. With a highly optimized supervised STDP rule we show that these accuracies can be achieved with just 20 epochs of training.

Kulkarni, Shruti↗
DOE OSTI2025

Efficient and Selective Chemical Transformations in Highly Charged and Confined Nanodroplets

The acceleration of chemical reaction rates and the increased product selectivity in microdroplets compared to that in bulk solutions has become a topic of increasing interest that has been extensively characterized by electrospray ionization mass spectrometry (ESI-MS). However, the sources of this acceleration and the detailed relationships between droplet properties and resulting reaction rate acceleration are still under debate. Moreover, droplet properties are governed by multiple interrelated experimental parameters, i.e., electrospray voltage, solution flow rate, etc., which makes it difficult and time-consuming to explore this diverse parameter space using traditional manual experimental or computational approaches. In this work, we developed an automated experimental platform integrating reactions in controlled charged microdroplet environments with ESI-MS characterization and sequential hybrid Bayesian modeling, as well as an optimal experimental design framework, to achieve multidimensional parameter optimization for higher reaction turnover rates, based on a model reaction of tetraethylenepentamine (TEPA) with carbon dioxide. With the current platform, we have achieved automated scans with a range of electrospray voltages and solution flow rates, and determined and optimized parameter settings to achieve increased reaction turnovers. We have also linked this platform to the underlying properties of droplets via a hybrid model incorporating physics, high-level theoretical calculations, and machine learning (ML) approaches. The autonomous platform is broadly applicable to a range of chemical reactions relevant to DOE’s mission in chemical separations, catalysis, and materials synthesis.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Generalized Bayesian MARS: Tools for Stochastic Computer Model Emulation

The multivariate adaptive regression spline (MARS) approach of Friedman and its Bayesian counterpart are effective approaches for the emulation of computer models. The traditional assumption of Gaussian errors limits the usefulness of MARS, and many popular alternatives, when dealing with stochastic computer models. Here, we propose a generalized Bayesian MARS (GBMARS) framework which admits the broad class of generalized hyperbolic distributions as the induced likelihood function. This allows us to develop tools for the emulation of stochastic simulators which are parsimonious, scalable, and interpretable and require minimal tuning, while providing powerful predictive and uncertainty quantification capabilities. GBMARS is capable of robust regression with t distributions, quantile regression with asymmetric Laplace distributions, and a general form of “Normal-Wald” regression in which the shape of the error distribution and the structure of the mean function are learned simultaneously. We demonstrate the effectiveness of GBMARS on various stochastic computer models, and we show that it compares favorably to several popular alternatives.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2014

A robust Bayesian approach to modeling epistemic uncertainty in common-cause failure models

In a standard Bayesian approach to the alpha-factor model for common-cause failure, a precise Dirichlet prior distribution models epistemic uncertainty in the alpha-factors. This Dirichlet prior is then updated with observed data to obtain a posterior distribution, which forms the basis for further inferences. In this paper, we adapt the imprecise Dirichlet model of Walley to represent epistemic uncertainty in the alpha-factors. In this approach, epistemic uncertainty is expressed more cautiously via lower and upper expectations for each alpha-factor, along with a learning parameter which determines how quickly the model learns from observed data. For this application, we focus on elicitation of the learning parameter, and find that values in the range of 1 to 10 seem reasonable. The approach is compared with Kelly and Atwood's minimally informative Dirichlet prior for the alpha-factor model, which incorporated precise mean values for the alpha-factors, but which was otherwise quite diffuse. Next, we explore the use of a set of Gamma priors to model epistemic uncertainty in the marginal failure rate, expressed via a lower and upper expectation for this rate, again along with a learning parameter. As zero counts are generally less of an issue here, we find that the choice of this learning parameter is less crucial. Finally, we demonstrate how both epistemic uncertainty models can be combined to arrive at lower and upper expectations for all common-cause failure rates. Thereby, we effectively provide a full sensitivity analysis of common-cause failure rates, properly reflecting epistemic uncertainty of the analyst on all levels of the common-cause failure model.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2024

Forward variable selection enables fast and accurate dynamic system identification with Karhunen-Loève decomposed Gaussian processes

A promising approach for scalable Gaussian processes (GPs) is the Karhunen-Loève (KL) decomposition, in which the GP kernel is represented by a set of basis functions which are the eigenfunctions of the kernel operator. Such decomposed kernels have the potential to be very fast, and do not depend on the selection of a reduced set of inducing points. However KL decompositions lead to high dimensionality, and variable selection thus becomes paramount. This paper reports a new method of forward variable selection, enabled by the ordered nature of the basis functions in the KL expansion of the Bayesian Smoothing Spline ANOVA kernel (BSS-ANOVA), coupled with fast Gibbs sampling in a fully Bayesian approach. It quickly and effectively limits the number of terms, yielding a method with competitive accuracies, training and inference times for tabular datasets of low feature set dimensionality. Theoretical computational complexities are O ( N P 2 ) in training and O ( P ) per point in inference, where N is the number of instances and P the number of expansion terms. The inference speed and accuracy makes the method especially useful for dynamic systems identification, by modeling the dynamics in the tangent space as a static problem, then integrating the learned dynamics using a high-order scheme. The methods are demonstrated on two dynamic datasets: a ‘Susceptible, Infected, Recovered’ (SIR) toy problem, along with the experimental ‘Cascaded Tanks’ benchmark dataset. Comparisons on the static prediction of time derivatives are made with a random forest (RF), a residual neural network (ResNet), and the Orthogonal Additive Kernel (OAK) inducing points scalable GP, while for the timeseries prediction comparisons are made with LSTM and GRU recurrent neural networks (RNNs) along with the SINDy package.

Hayes, Kyle↗
DOE OSTI2020

An active learning high-throughput microstructure calibration framework for solving inverse structure–process problems in materials informatics

Determining a process–structure–property relationship is the holy grail of materials science, where both computational prediction in the forward direction and materials design in the inverse direction are essential. Problems in materials design are often considered in the context of process–property linkage by bypassing the materials structure, or in the context of structure–property linkage as in microstructure-sensitive design problems. However, there is a lack of research effort in studying materials design problems in the context of process–structure linkage, which has a great implication in reverse engineering. In this paper, given a target microstructure, we propose an active learning high-throughput microstructure calibration framework to derive a set of processing parameters, which can produce an optimal microstructure that is statistically equivalent to the target microstructure. The proposed framework is formulated as a noisy multi-objective optimization problem, where each objective function measures a deterministic or statistical difference of the same microstructure descriptor between a candidate microstructure and a target microstructure. Furthermore, to significantly reduce the physical waiting wall-time, we enable the high-throughput feature of the microstructure calibration framework by adopting an asynchronously parallel Bayesian optimization to exploit high-performance computing resources. Case studies in additive manufacturing and grain growth are used to demonstrate the applicability of the proposed framework, where kinetic Monte Carlo (kMC) simulation is used as a forward predictive model, such that for a given target microstructure, the target processing parameters that produced this microstructure are successfully recovered.

36 MATERIALS SCIENCE↗