Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian Optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 595 records · Page 33
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Marginal unbiased score expansion and application to CMB lensing

Here, we present the marginal unbiased score expansion (MUSE) method, an algorithm for generic high-dimensional hierarchical Bayesian inference. MUSE performs approximate marginalization over arbitrary non-Gaussian latent parameter spaces, yielding Gaussianized asymptotically unbiased and near-optimal constraints on global parameters of interest. It is computationally much cheaper than exact alternatives like Hamiltonian Monte Carlo (HMC), excelling on funnel problems which challenge HMC, and does not require any problem-specific user supervision like other approximate methods such as variational inference or many simulation-based inference methods. MUSE makes possible the first joint Bayesian estimation of the delensed Cosmic Microwave Background (CMB) power spectrum and gravitational lensing potential power spectrum, demonstrated here on a simulated data set as large as the upcoming South Pole Telescope 3G 1500 deg 2 survey, corresponding to a latent dimensionality of ~6 million and of order 100 global bandpower parameters. On a subset of the problem where an exact but more expensive HMC solution is feasible, we verify that MUSE yields nearly optimal results. We also demonstrate that existing spectrum-based forecasting tools which ignore pixel-masking underestimate predicted error bars by only ~10%. This method is a promising path forward for fast lensing and delensing analyses which will be necessary for future CMB experiments such as SPT-3G, Simons Observatory, or CMB-S4, and can complement or supersede existing HMC approaches. The success of MUSE on this challenging problem strengthens its case as a generic procedure for a broad class of high-dimensional inference problems.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Bayesian inferences of electrical current delivered to shocked transmission lines

Small radius Bdot measurements of electrical current delivered by pulsed power devices are routinely compromised by electrode/convolute plasmas endemic to multiterawatt transmission lines. Inferences of delivered current unaffected by these issues have recently been obtained by numerically optimizing consistency between model predictions and local experimental velocimetry data, but these are only unique for shockless velocity profiles. In the paper we describe a more general Bayesian method capable of inferring current despite the presence of shocks. Additionally, we describe uncertainty estimates and use of the technique on experimental data. This technique is the first to provide uncertainty estimates on the full current trace delivered to an inertial confinement fusion target.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

Online LIBS–ML Framework for Dynamic Characterization of Heterogeneous Waste-Derived Gasification Feedstocks

LIBS−ML framework for real time feedstock characterization during continuous conveyor transport Heterogeneous waste derived feedstocks (e.g., waste coal, biomass and blends) introduce rapid variability in heating value and ash chemistry that affect gasifier operation, yet conventional laboratory characterization techniques are too slow to support proactive control. To address this gap, this study reports on an online, in situ, dynamic characterization framework that couple’s laser-induced breakdown spectroscopy (LIBS) with leakage safe machine learning (ML) regression to deliver real time, decision quality predictions of gasifier relevant properties. A controlled sample matrix spanning two different waste coals, two different biomasses, and engineered blends under two particle size conditions were constructed and benchmarked using standardized laboratory analyses for proximate/ultimate properties and ash composition. LIBS spectra were acquired dynamically as material flowed on a conveyor belt, using high energy 1064 nm laser ablation and shot averaging to improve repeatability and precision. Supervised regression models (multi layer perceptron (MLP) /artificial neural network (ANN), random forest (RF), and support vector regression (SVR)) and an optimized weighted ensemble were trained on emission line feature sets using nested cross validation with Bayesian hyperparameter tuning and validated against an independent hold out set. The proposed LIBS−ML workflow achieves near laboratory predictive fidelity across parametric targets (including higher heating value (HHV), ash content, fixed carbon, sulfur, major ash forming oxides, and initial deformation temperature (IDT)), with the weighted ensemble providing a robust default predictor under dynamic measurement conditions. These results demonstrate a practical pathway for real time feedstock characterization that can enable feedforward adjustments and more resilient gasifier operation for variable quality waste derived fuels.

Biomass↗
DOE OSTI2021

Adaptive hyperparameter updating for training restricted Boltzmann machines on quantum annealers

Restricted Boltzmann Machines (RBMs) have been proposed for developing neural networks for a variety of unsupervised machine learning applications such as image recognition, drug discovery, and materials design. The Boltzmann probability distribution is used as a model to identify network parameters by optimizing the likelihood of predicting an output given hidden states trained on available data. Training such networks often requires sampling over a large probability space that must be approximated during gradient based optimization. Quantum annealing has been proposed as a means to search this space more efficiently which has been experimentally investigated on D-Wave hardware. D-Wave implementation requires selection of an effective inverse temperature or hyperparameter (β) within the Boltzmann distribution which can strongly influence optimization. Here, we show how this parameter can be estimated as a hyperparameter applied to D-Wave hardware during neural network training by maximizing the likelihood or minimizing the Shannon entropy. We find both methods improve training RBMs based upon D-Wave hardware experimental validation on an image recognition problem. Neural network image reconstruction errors are evaluated using Bayesian uncertainty analysis which illustrate more than an order magnitude lower image reconstruction error using the maximum likelihood over manually optimizing the hyperparameter. The maximum likelihood method is also shown to out-perform minimizing the Shannon entropy for image reconstruction.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

From Optimization to Sampling Through Gradient Flows

Optimization and sampling algorithms play a central role in science and engineering as they enable finding optimal predictions, policies, and recommendations, as well as expected and equilibrium states of complex systems. The notion of “optimality” is formalized by the choice of an objective function, while the notion of an “expected” state is specified by a probabilistic model for the distribution of states. Optimizing rugged objective functions and sampling multimodal distributions is computationally challenging, especially in high-dimensional problems. Here, for this reason, many optimization and sampling methods have been developed by researchers working in disparate fields such as Bayesian statistics, molecular dynamics, genetics, quantum chemistry, machine learning, weather forecasting, econometrics, and medical imaging.

Trillos, N. García↗
DOE OSTI2022

GIGA-Lens: Fast Bayesian Inference for Strong Gravitational Lens Modeling

We present GIGA-Lens: a gradient-informed, GPU-accelerated Bayesian framework for modeling strong gravitational lensing systems, implemented in TensorFlow and JAX. The three components, optimization using multistart gradient descent, posterior covariance estimation with variational inference, and sampling via Hamiltonian Monte Carlo, all take advantage of gradient information through automatic differentiation and massive parallelization on graphics processing units (GPUs). We test our pipeline on a large set of simulated systems and demonstrate in detail its high level of performance. The average time to model a single system on four Nvidia A100 GPUs is 105 s. The robustness, speed, and scalability offered by this framework make it possible to model the large number of strong lenses found in current surveys and present a very promising prospect for the modeling of ${ \mathcal O }({10}^{5})$ lensing systems expected to be discovered in the era of the Vera C. Rubin Observatory, Euclid, and the Nancy Grace Roman Space Telescope.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Extending Parsimonious Bayesian Inference

Parsimonious Bayesian inference is a theoretical framework for efficient data assimilation that seeks to balance increased consistency between predictions and training data against corresponding increases in model complexity. Within this framework, over-training is understood as optimization that encodes excessive information within model parameters while only achieving small improvements between predictions and training data. This project aims to develop practical methods of limiting excess model information during optimization. One key observation is that practical heuristics for parsimonious learning in high-dimensions must balance expressivity, i.e. the ability of the model to capture diverse predictions with only a few non-zero parameters, against discoverability, i.e. the ability to train the model with gradient-based optimization and drive parameters to low information states. As such, we developed logical activation functions that are able to adaptively approximate arbitrary truth tables that define Boolean logic operations within a probabilistic framework. These functions have demonstrated the ability to learn exclusive disjunction (XOR) and conditioned disjunction (if [condition] then [result_if_true] else [result_if_false]) within a single layer of a neural network. To efficiently exploit these activation functions to drive parsimonious learning required several other advances within the domain of variational inference. The most efficient form of complexity suppression is structured sparsification, driving most model parameters to zero while achieving the structural coherence among nonzeros needed for bandwidth reduction. Such models are not only far more efficient at suppressing information-theoretic complexity, they also reduce the other forms of complexity (computations, communication, storage, and the number of dependencies needed to evaluate predictions). Aiming to support enhanced sparsification, this project examined new approaches to high-dimensional variational inference that allow us to calibrate and control parameter uncertainty during optimization. By identifying which parameters can sustain sparsifying perturbations with little impact on prediction quality, we can develop better pruning strategies by framing them as approximate Bayesian inference. These advances also open paths to mitigate concerns with deploying advanced learning methods in resource-constrained environments, such as running models on power-limited or communication-limited devices.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Statistical Treatment of Inverse Problems Constrained by Differential Equations-based Models with Stochastic Terms

This paper introduces a statistical treatment of inverse problems constrained by models with stochastic terms. The solution of the forward problem is given by a distribution represented numerically by an ensemble of simulations. The goal is to formulate the inverse problem, in particular the objective function, to find the closest forward distribution (i.e., the output of the stochastic forward problem) that best explains the distribution of the observations in a certain metric. We use proper scoring rules, a concept employed in statistical forecast verification, namely energy, variogram, and hybrid (i.e., combination of the two) scores. We study the performance of the proposed formulation in the context of two applications: a coefficient field inversion for subsurface flow governed by an elliptic partial differential equation with a stochastic source and a parameter inversion for power grid governed by differential-algebraic equations. In both cases we show that the variogram and the hybrid scores produce better parameter inversion results than does the energy score, whereas the energy score leads to better probabilistic predictions.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1986

Application of physical parameter identification to finite element models

A time domain technique for matching response predictions of a structural dynamic model to test measurements is developed. Significance is attached to prior estimates of physical model parameters and to experimental data. The Bayesian estimation procedure allows confidence levels in predicted physical and modal parameters to be obtained. Structural optimization procedures are employed to minimize an error functional with physical model parameters describing the finite element model as design variables. The number of complete FEM analyses are reduced using approximation concepts, including the recently developed convoluted Taylor series approach. The error function is represented in closed form by converting free decay test data to a time series model using Prony' method. The technique is demonstrated on simulated response of a simple truss structure.

Bronowicki, Allen J.↗
DOE OSTI2021

Sparse Bayesian mass mapping with uncertainties: hypothesis testing of structure

ABSTRACT A crucial aspect of mass mapping, via weak lensing, is quantification of the uncertainty introduced during the reconstruction process. Properly accounting for these errors has been largely ignored to date. We present a new method to reconstruct maximum a posteriori (MAP) convergence maps by formulating an unconstrained Bayesian inference problem with Laplace-type l1-norm sparsity-promoting priors, which we solve via convex optimization. Approaching mass mapping in this manner allows us to exploit recent developments in probability concentration theory to infer theoretically conservative uncertainties for our MAP reconstructions, without relying on assumptions of Gaussianity. For the first time, these methods allow us to perform hypothesis testing of structure, from which it is possible to distinguish between physical objects and artefacts of the reconstruction. Here, we present this new formalism, and demonstrate the method on simulations, before applying the developed formalism to two observational data sets of the Abell 520 cluster. Initial reconstructions of the Abell 520 catalogues reported the detection of an anomalous ‘dark core’ – an overdense region with no optical counterpart – which was taken to be evidence for self-interacting dark matter. In our Bayesian framework, it is found that neither Abell 520 data set can conclusively determine the physicality of such dark cores at $99{{\ \rm per\ cent}}$ confidence. However, in both cases the recovered MAP estimators are consistent with both sets of data.

Price, M. A.↗
DOE OSTI2022

Kinetic model development and Bayesian uncertainty quantification for the complete reduction of Fe-based oxygen carriers with CH 4 , CO, and H 2 for chemical looping combustion

In this work, three kinetic models are developed and calibrated for the complete multi-step reduction of an Fe-based oxygen carrier (OC) particle with CH 4 , CO, and H 2 , using data from thermogravimetric analysis. The complete reduction rate profiles exhibit complex dynamics whose trajectory is significantly different depending on the reducing gas. A Bayesian model building and parameter estimation framework is applied for simultaneous parameter and model structure uncertainty quantification. The final models show excellent agreement between model predictions and calibration data, as well as new data not used for calibration (for the reduction of the OC with CH 4 ). Parameter uncertainty is quantified by determining joint posterior distribution, and model structure uncertainty is addressed by incorporating Gaussian process stochastic functions (represented by Bayesian smoothing splines) into the kinetic models. The final kinetic models with discrepancy functions are readily employable in equation-oriented simulation and optimization platforms.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
NASA NTRS2018

Automated Pneumothorax Diagnosis using Deep Neural Networks

Thoracic ultrasound can provide information leading to rapid diagnosis of pneumothorax with improved accuracy over the standard physical examination and with higher sensitivity than anteroposterior chest radiography. However, the clinical We have Furthermore, remote environments, such as the battlefield or deep-space exploration, may lack expertise for diagnosing developed an automated image interpretation pipeline for the analysis of thoracic ultrasound data and the classification of pneumothorax events to provide decision support in such situations. Our pipeline consists of image preprocessing, data augmentation, and deep learning architectures for medical diagnosis. In this work, we demonstrate that robust, accurate interpretation of chest images and video can be achieved using deep neural networks. A number of novel image processing techniques were employed to achieve this result. Affine transformations were applied for data augmentation. Hyperparameters were optimized for learning rate, dropout regularization, batch size, and epoch iteration by a sequential model-based Bayesian approach. In addition, we utilized pretrained architecturesinterpretation of a patient medical image is highly operator dependent. certain pathologies., applying transfer learning and fine-tuning techniques to fully connected layers. Our pipeline yielded binary classification validation accuracies of 98.3% for M-mode images and 99.8% with B-mode video frames.

US Army collaboration↗
NASA NTRSDate not supplied

Developing a Genetic Variant Calling Pipeline for Quantifying the Complex Mutagenic Load Accumulated in BioNutrients-1 Production Pack Samples

Microorganisms hold great promise for on demand production of labile nutrients and pharmaceuticals as well recycling and in situ resource utilization. The utilization of microorganisms for such tasks on space missions is hindered by the limited data on how microbes respond to spaceflight. For example, the genetic stability of microorganisms, and the genomic engineered traits added to deliver desired functions, over long-term storage in the spacecraft environment is poorly understood. The BioNutrients-1 (BN-1) mission conducted a 5-year study of desiccated storage in Low Earth Orbit (LEO) to evaluate the suitability of eight synthetic biology chassis organisms for long-duration space missions. We are employing high-depth, whole genome sequencing (WGS) to determine the mutagenic load that accumulated during long-term storage. Mutation analysis pipelines are well established for homogenous culture grown from a single colony, but the mutational landscape of the BN-1 samples present a unique analysis challenge, as every cell in the BN-1 samples had a unique genetic journey of DNA damage and repair. Consequently, sequence variants are expected at low allele frequency within samples. To address this genetic complexity, we apply two distinct computational approaches to identify mutations in pre-existing WGS data collected from populations of Chlamydomonas reinhardtii that were exposed to UV mutagenesis and growth in LEO. For reference genome free mutation detection, we utilized DiscoSNP++, which is a de Bruijn graph approach. For reference genome-based mutation detection we utilize GATK for Microbes, which is a Bayesian probabilistic approach. We will benchmark these approaches against the mutations originally identified using CRISP, a method optimized for pooled samples. Ultimately, quantifying the mutation load imposed by storage or growth on the ISS will help identify chassis organisms with both high levels of genome stability and viability, which are desirable traits for implementation of bioproduction in long-duration missions.

SNP↗
DOE OSTI2025

Quantifying bias due to non-Gaussian foregrounds in an optimal reconstruction of CMB lensing and temperature power spectra

We estimate the magnitude of the bias due to non-Gaussian extragalactic foregrounds on the optimal reconstruction of the cosmic microwave background (CMB) lensing potential and temperature power spectra. The reconstruction is performed using a Bayesian inference method known as the marginal unbiased score expansion (MUSE). We apply MUSE to a minimum variance combination of multifrequency maps drawn from the Agora publicly available simulations of the lensed CMB and correlated extragalactic foreground emission. Taking noise levels appropriate to the SPT-3G D1 release, we find non-Gaussian foregrounds may bias the MUSE reconstruction of the lensing potential amplitude at the level of (0.7 ± 0.3)σ when using modes up to ℓ max = 3500. We do not detect a statistically significant bias, finding a value of (-0.4 ± 0.3)σ, when restricted to lower angular multipoles, ℓ max = 3000. This work is a first step toward understanding the impact of extragalactic foregrounds on optimal reconstructions of CMB temperature and lensing potential power spectra.

Statistical sampling techniques↗
DOE OSTI2022

Automatic Calibration of a Geomechanical Model from Sparse Data for Estimating Stress in Deep Geological Formations

Summary In this study, we demonstrate geomechanical modeling with fully automatic parameter calibration to estimate the full geomechanical stress fields of a prospective US carbon dioxide (CO2) storage site, based on sparse measurement data. The goal is to compute full stress tensor field estimates (principal stresses and orientations) that are maximally compatible with observations within the constraints of the model assumptions, thereby extending pointwise, incomplete partial stress measurement to a simulated full formation stress field, as well as a rough assessment of the associated error. We use the Perch site, located in Otsego County, Michigan, USA, as our case study. The input data consist of partial stress tensor information inferred from in-situ borehole tests, geophysical well logs, and processing of seismic data. A static earth model (SEM) of the site was developed, and geomechanical simulation functionality of the open-source MATLAB Reservoir Simulation Toolbox (MRST) was used to model the stress field. Adjoint-based nonlinear optimization was used to adjust boundary conditions and material properties to calibrate simulated results of observations. Results were interpreted through a Bayesian framework. The focus of this paper is to demonstrate how the fully automatic calibration procedure works and discuss the results obtained; it does not attempt a detailed analysis of the stress field in the context of the proposed CO2 storage initiatives. Our work is part of a larger effort to noninvasively determine in-situ stresses in deep formations considered for CO2 storage. Guided by previously published research on geomechanical model calibration, our work presents a novel calibration approach supporting a potentially large number of linear or nonlinear calibration parameters to produce results optimally agreeing with available measurements and thus extend partial pointwise estimates to full tensor fields compatible with the physics of the site.

Engineering↗
DOE OSTI2023

Massively Parallel Bayesian Model Calibration and Uncertainty Quantification with Applications to Nuclear Fuels and Materials

The U.S. Department of Energy (DOE)’s Nuclear Energy Advanced Modeling and Simulation (NEAMS) program aims to develop predictive capabilities by applying computational methods to the analysis and design of advanced reactor and fuel cycle systems. This program has been providing engineering-scale support for the development of BISON, a high-fidelity and high-resolution fuel performance tool. Fuel behavior in a nuclear reactor is governed by a complex network of mechanisms interacting with various other physics aspects in the reactor system. Any model developed to represent the fuel behavior will likely be idealized resulting in uncertainties in their predictions compared to the observed data. As such, this report was motivated by the need to identify the sources of uncertainties and quantify and propagate them through the fuel model outputs. Such quantification of uncertainties will establish a level of model trustworthiness, identify approaches to improve the model trustworthiness, and even guide optimal experiment design for maximal information gain. To accomplish the uncertainty quantification for computational models, this report has relied on the Bayesian framework which provides probabilistic treatment of models their inputs and outputs. The current state-of-the-art on performing Bayesian Uncertainty Quantification (UQ) for nuclear engineering models using High Performance Computing (HPC) resources have been reviewed. Implementation of capabilities for massively parallel Bayesian UQ in Multiphysics Object-Oriented Simulation Environment (MOOSE) is discussed. Several verification cases are discussed to verify the accuracy of the quantified uncertainties using the developed computational capabilities in MOOSE. Then, the problem of quantifying the uncertainties in TRI-Structural isOtropic (TRISO) fuel silver release is addressed. For the first time, the uncertainties arising from the TRISO Fission Gas Release (FGR) model due to model inadequacy and experimental noise are quantified. Also, the Bayesian capabilities are applied to the calibration of the MATPRO creep model, a widely used model in several fuel assessment cases. The impact of the prediction uncertainties in the MATPRO model on the fuel cladding behavior as part of the TRIBULATION assessment case (which is an integral effects case) is investigated. This report concludes with a discussion on the future work for the UQ for computational models.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2022

A Logarithmic Bayesian Approach to Quantum Error Detection

We consider the problem of continuous quantum error correction from a Bayesian perspective, proposing a pair of digital filters using logarithmic probabilities that are able to achieve near-optimal performance on a three-qubit bit-flip code, while still being reasonable to implement on low-latency hardware. These practical filters are approximations of an optimal filter that we derive explicitly for finite time steps, in contrast with previous work that has relied on stochastic differential equations such as the Wonham filter. By utilizing logarithmic probabilities, we are able to eliminate the need for explicit normalization and can reduce the Gaussian noise distribution to a simple quadratic expression. The state transitions induced by the bit-flip errors are modeled using a Markov chain, which for log-probabilities must be evaluated using a LogSumExp function. We develop the two versions of our filter by constraining this LogSumExp to have either one or two inputs, which favors either simplicity or accuracy, respectively. Using simulated data, we demonstrate that the single-term and two-term filters are able to significantly outperform both a double threshold scheme and a linearized version of the Wonham filter in tests of error detection under a wide variety of error rates and time steps.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS1975

Adaptive statistical pattern classifiers for remotely sensed data

A technique for the adaptive estimation of nonstationary statistics necessary for Bayesian classification is developed. The basic approach to the adaptive estimation procedure consists of two steps: (1) an optimal stochastic approximation of the parameters of interest and (2) a projection of the parameters in time or position. A divergence criterion is developed to monitor algorithm performance. Comparative results of adaptive and nonadaptive classifier tests are presented for simulated four dimensional spectral scan data.

Gonzalez, R. C.↗