Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian Optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 631 records · Page 35
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

Towards optimal extraction of cosmological information from nonlinear data

One of the main unsolved problems of cosmology is how to maximize the extraction of information from nonlinear data. If the data are nonlinear the usual approach is to employ a sequence of statistics (N-point statistics, counting statistics of clusters, density peaks or voids etc.), along with the corresponding covariance matrices. However, this approach is computationally prohibitive and has not been shown to be exhaustive in terms of information content. Here we instead develop a hierarchical Bayesian approach, expanding the likelihood around the maximum posterior of linear modes, which we solve for using optimization methods. By integrating out the modes using perturbative expansion of the likelihood we construct an initial power spectrum estimator, which for a fixed forward model contains all the cosmological information if the initial modes are gaussian distributed. We develop a method to construct the window and covariance matrix such that the estimator is explicitly unbiased and nearly optimal. We then generalize the method to include the forward model parameters, including cosmological and nuisance parameters, and primordial non-gaussianity. We apply the method in the simplified context of nonlinear structure formation, using either simplified 2-LPT dynamics or N-body simulations as the nonlinear mapping between linear and nonlinear density, and 2-LPT dynamics in the optimization steps used to reconstruct the initial density modes. We demonstrate that the method gives an unbiased estimator of the initial power spectrum, providing among other a near optimal reconstruction of linear baryonic acoustic oscillations.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Multi-Modal Bayesian Neural Network Surrogates with Conjugate Last-Layer Estimation

As data collection and simulation capabilities advance, multi-modal learning, the task of learning from multiple modalities and sources of data, is becoming an increasingly important area of research. Surrogate models that learn from data of multiple auxiliary modalities to support the modeling of a highly expensive quantity of interest have the potential to aid outer loop applications such as optimization, inverse problems, or sensitivity analyses when multi-modal data are available. We develop two multi-modal Bayesian neural network surrogate models and leverage conditionally conjugate distributions in the last layer to estimate model parameters using stochastic variational inference (SVI). We provide a method to perform this conjugate SVI estimation in the presence of partially missing observations. Here, we demonstrate improved prediction accuracy and uncertainty quantification compared to unimodal surrogate models for both scalar and time series data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Bayesian projection pursuit regression

In projection pursuit regression (PPR), a univariate response variable is approximated by the sum of $M$ “ridge functions,” which are flexible functions of one-dimensional projections of a multivariate input variable. Traditionally, optimization routines are used to choose the projection directions and ridge functions via a sequential algorithm, and $M$ is typically chosen via cross-validation. Here, we introduce a novel Bayesian version of PPR, which has the benefit of accurate uncertainty quantification. To infer appropriate projection directions and ridge functions, we apply novel adaptations of methods used for the single ridge function case ($M$=1), called the Bayesian Single Index Model; and use a Reversible Jump Markov chain Monte Carlo algorithm to infer the number of ridge functions $M$. We evaluate the predictive ability of our model in 20 simulated scenarios and for 23 real datasets, in a bake-off against an array of state-of-the-art regression methods. Finally, we generalize this methodology and demonstrate the ability to accurately model multivariate response variables. Its effective performance indicates that Bayesian Projection Pursuit Regression is a valuable addition to the existing regression toolbox.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Coupled Investigation of Fracture Permeability Impact on Reservoir Stress and Seismic Slip Behavior

Our goal is to develop, apply and validate a holistic thermal, hydrologic, mechanical, and chemical (THMC) workflow that includes evaluation of induced seismic slip in EGS reservoirs. We will integrate experimental and modeling approaches to reduce parameter uncertainty and better predict/mitigate seismic hazard at EGS sites. Our novel approach couples 3D physics-based earthquake simulations with THMC models (THMc+E). This capability will enable improve engineering decisions at Utah-FORGE and move EGS operations toward repeatable, robust, economically viable, and socially accepted development. For example, our THMC+E models will predict circulation scenarios and related seismic hazard for a suite of flow rates and under uncertainty, thus enabling evaluation of optimal circulation strategy. Laboratory experiments will be performed to constrain key model parameters and Bayesian techniques will provide a probabilistic evaluation of parameters used in models. THMC+E simulations will enable exploration various circumstances that may hinder EGS success and develop mitigation strategies.

58 GEOSCIENCES↗
DOE OSTI2024

5-2428: Fracture Permeability Impact on Seismic Slip Behavior

Our goal is to develop, apply and validate a holistic thermal, hydrologic, mechanical, and chemical (THMC) workflow that includes evaluation of induced seismic slip in EGS reservoirs. We will integrate experimental and modeling approaches to reduce parameter uncertainty and better predict/mitigate seismic hazard at EGS sites. Our novel approach couples 3D physics-based earthquake simulations with THMC models (THMc+E). This capability will enable improve engineering decisions at Utah-FORGE and move EGS operations toward repeatable, robust, economically viable, and socially accepted development. For example, our THMC+E models will predict circulation scenarios and related seismic hazard for a suite of flow rates and under uncertainty, thus enabling evaluation of optimal circulation strategy. Laboratory experiments will be performed to constrain key model parameters and Bayesian techniques will provide a probabilistic evaluation of parameters used in models. THMC+E simulations will enable exploration various circumstances that may hinder EGS success and develop mitigation strategies.

58 GEOSCIENCES↗
DOE OSTI2023

Non-commutative disintegrations: Existence and uniqueness in finite dimensions

Motivated by advances in categorical probability, we introduce non-commutative almost everywhere (a.e.) equivalence and disintegrations in the setting of C*-algebras. We show that C*-algebras (resp. W*-algebras) and a.e. equivalence classes of 2-positive (resp. positive) unital maps form a category. We prove non-commutative disintegrations are a.e. unique whenever they exist. We provide an explicit characterization for when disintegrations exist in the setting of finite-dimensional C*-algebras, and we give formulas for the associated disintegrations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Efficient and choreographed quality-of- service management in dense 6G verticals with high-speed mobility requirements

Future 6G networks are envisioned to support very heterogeneous and extreme applications (known as verticals). Some examples are further-enhanced mobile broadband communications, where bitrates could go above one terabit per second, or extremely reliable and low-latency communications, whose end-to-end delay must be below one hundred microseconds. To achieve that ultra-high Quality-of-Service, 6G networks are commonly provided with redundant resources and intelligent management mechanisms to ensure that all devices get the expected performance. But this approach is not feasible or scalable for all verticals. Specifically, in 6G scenarios, mobile devices are expected to have speeds greater than 500 kilometers per hour, and device density will exceed ten million devices per square kilometer. In those verticals, resources cannot be redundant as, because of such a huge number of devices, Quality-of-Service requirements are pushing the effective performance of technologies at physical level. And, on the other hand, high-speed mobility prevents intelligent mechanisms to be useful, as devices move around and evolve faster than the usual convergence time of those intelligent solutions. New technologies are needed to fill this unexplored gap. Therefore, in this paper we propose a choreographed Quality-of-Service management solution, where 6G base stations predict the evolution of verticals at real-time, and run a lightweight distributed optimization algorithm in advance, so they can manage the resource consumption and ensure all devices get the required Quality-of-Service. Prediction mechanism includes mobility models (Markov, Bayesian, etc.) and models for time-variant communication channels. Besides, a traffic prediction solution is also considered to explore the achieved Quality-of-Service in advance. The optimization algorithm calculates an efficient resource distribution according to the predicted future vertical situation, so devices achieve the expected Quality-of-Service according to the proposed traffic models. An experimental validation based on simulation tools is also provided. Results show that the proposed approach reduces up to 12% of the network resource consumption for a given Quality-of-Service.

42 ENGINEERING↗
DOE OSTI2021

Exploring the physics of cesium lead halide perovskite quantum dots via Bayesian inference of the photoluminescence spectra in automated experiment

The unique optoelectronic properties of metal halide perovskite quantum dots (QDs) make them promising candidates for applications in light-emitting diodes (LEDs), scintillators, and other photonic devices. The automated micropipetting synthesis platform equipped with an optical reader enables the opportunity for high throughput synthesis and photoluminescent (PL) characterization of metal halide perovskite QDs for the first time. Here, we explore the compositional dependence of the PL behavior and stability of the combinatorial library of cesium lead halide (CsPbX 3 ) perovskites QDs via the automated platform. To study the stability of synthesized QDs in the binary and ternary configurations, we study the time-dependent PL properties using previously developed machine learning analysis. Additionally, to systematically explore the PL behavior in the ternary CsPbX 3 QDs system, we introduce the Bayesian inference framework that allows the probabilistic fit of multiple models to the PL data and establishes both optimal model and model parameter robustly. Furthermore, these behaviors can be used as a control parameter for the navigation of the multidimensional compositional spaces in automated synthesis. Furthermore, this analysis shows the nonuniformity of the PL peak behavior in the ternary CsPbX 3 QDs system. Further, the analysis confirms narrow size distribution and good quality of CsPbBr 3 QDs alloyed with low concentrations of iodide and chloride. We note that Bayesian Inference fit parameters can be further used as a control signal for navigation of the chemical spaces in automated synthesis.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Marginal unbiased score expansion and application to CMB lensing

Here, we present the marginal unbiased score expansion (MUSE) method, an algorithm for generic high-dimensional hierarchical Bayesian inference. MUSE performs approximate marginalization over arbitrary non-Gaussian latent parameter spaces, yielding Gaussianized asymptotically unbiased and near-optimal constraints on global parameters of interest. It is computationally much cheaper than exact alternatives like Hamiltonian Monte Carlo (HMC), excelling on funnel problems which challenge HMC, and does not require any problem-specific user supervision like other approximate methods such as variational inference or many simulation-based inference methods. MUSE makes possible the first joint Bayesian estimation of the delensed Cosmic Microwave Background (CMB) power spectrum and gravitational lensing potential power spectrum, demonstrated here on a simulated data set as large as the upcoming South Pole Telescope 3G 1500 deg 2 survey, corresponding to a latent dimensionality of ~6 million and of order 100 global bandpower parameters. On a subset of the problem where an exact but more expensive HMC solution is feasible, we verify that MUSE yields nearly optimal results. We also demonstrate that existing spectrum-based forecasting tools which ignore pixel-masking underestimate predicted error bars by only ~10%. This method is a promising path forward for fast lensing and delensing analyses which will be necessary for future CMB experiments such as SPT-3G, Simons Observatory, or CMB-S4, and can complement or supersede existing HMC approaches. The success of MUSE on this challenging problem strengthens its case as a generic procedure for a broad class of high-dimensional inference problems.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2023

Bayesian inferences of electrical current delivered to shocked transmission lines

Small radius Bdot measurements of electrical current delivered by pulsed power devices are routinely compromised by electrode/convolute plasmas endemic to multiterawatt transmission lines. Inferences of delivered current unaffected by these issues have recently been obtained by numerically optimizing consistency between model predictions and local experimental velocimetry data, but these are only unique for shockless velocity profiles. In the paper we describe a more general Bayesian method capable of inferring current despite the presence of shocks. Additionally, we describe uncertainty estimates and use of the technique on experimental data. This technique is the first to provide uncertainty estimates on the full current trace delivered to an inertial confinement fusion target.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2016

Learning Weight Uncertainty with Stochastic Gradient MCMC for Shape Classification

Learning the representation of shape cues in 2D & 3D objects for recognition is a fundamental task in computer vision. Deep neural networks (DNNs) have shown promising performance on this task. Due to the large variability of shapes, accurate recognition relies on good estimates of model uncertainty, ignored in traditional training of DNNs, typically learned via stochastic optimization. This paper leverages recent advances in stochastic gradient Markov Chain Monte Carlo (SG-MCMC) to learn weight uncertainty in DNNs. It yields principled Bayesian interpretations for the commonly used Dropout/DropConnect techniques and incorporates them into the SG-MCMC framework. Extensive experiments on 2D & 3D shape datasets and various DNN models demonstrate the superiority of the proposed approach over stochastic optimization. Our approach yields higher recognition accuracy when used in conjunction with Dropout and Batch-Normalization.

convolutional neural network↗
DOE OSTI2017

Preface to the Special Issue on TOUGH Symposium 2015

The TOUGH Symposium 2015 was held in Berkeley, California, September 28-30, 2015. The TOUGH family of codes, developed at the Energy Geosciences Division of Lawrence Berkeley National Laboratory (LBNL), is a suite of computer programs for the simulation of multiphase and multicomponent fluid and heat flows in porous and fractured media with applications in many geosciences fields, such as geothermal reservoir engineering, nuclear waste disposal, geological carbon sequestration, oil and gas reservoirs, gas hydrate research, vadose zone hydrology and environmental remediation. Since the first release in the 1980s, many modifications and enhancements have been continuously made to TOUGH and its various descendants (iTOUGH2, TOUGH+, TOUGH-MP, TOUGHREACT, TOUGH+HYDRATE, TMVOC...), at LBNL and elsewhere. Today, these codes are used worldwide in academia, government organizations and private companies in problems involving coupled hydrological, thermal, biogeochemical and geomechanical processes. The Symposia, organized every 2-3 years, bring together developers and users for an open exchange on recent code enhancements and applications. In 2015, the Symposium was attended by one hundred participants, representing thirty-four nationalities. This Special Issue in Computers & Geosciences gathers extended versions of selected Symposium proceedings related to (i) recent enhancements to the TOUGH family of codes and (ii) coupled flow and geomechanics processes modeling. In this special issue, a paper by Jung et al. provides a new base version of TOUGH, TOUGH3. This latest version addresses the computational robustness and efficiency needed for the increasing complexity of the simulated processes and the growing size of model domains. Moreover, it consolidates the serial and parallel versions into a single code, hence, inconsistencies between different versions are less likely to occur, and code maintenance and development become more efficient. Finsterle et al. present a review of the capabilities of iTOUGH2 and illustrate the workflow of an integrated simulation-optimization framework. Liu et al. present an approach to reduce the computational cost of performing Bayesian inverse modeling and demonstrate their approach using a synthetic experiment simulated with TOUGH2. Regarding the mesh generation process, the paper by Doughty presents a generalization of the grid-generation algorithm embedded in TOUGH2 (MESHMAKER) to create one-column grids with fractal dimension, useful for representing flow through fractures or highly heterogeneous media with fractal geometry, and also for inverse methods. Sentís and Gable present a module to interface the open source code Lagrit with TOUGH2, that assists in the processes of mesh generation and model setup, and that can be particularly useful for complex geometries and large models. Bonduà et al. present a pre-processing tool (VORO2MESH) that features a full 3D Voronoi approach for structured and unstructured grids, and an enhanced post-processor (TOUGH2Viewer) for the visualization of 3D Voronoi grids. The literature review shows that coupled fluid flow and geomechanics has gained increasing attention in the last years in many geosciences fields. In this context, several codes of the TOUGH family have been successfully coupled with geomechanics software. The paper by Rutqvist presents a review of some of these TOUGH-based geomechanics models, of which the TOUGH-FLAC simulator – presented for the first time in the TOUGH Symposium 2003 – is well-known worldwide and has been used in a large variety of applications, encompassing the application domains of the TOUGH codes. Blanco-Martín et al. present an extension of TOUGH-FLAC to the finite strain framework, motivated by the existence of large strains associated with rock salt deformation in the context of heat-generating nuclear waste disposal. Following a paper published in the Special Issue of the TOUGH Symposium 2012 in this journal, Kim et al. present the current status of the TOUGH-RSBN simulator for coupled hydro-mechanical modeling of hydraulic fracturing using a discrete fracture network approach, and present some validation and demonstration cases. Rinaldi and Nespoli present TOUGH-SEED, an injection-induced seismicity modeling approach using TOUGH2 and a geomechanics stochastic model, and apply it to the Basel EGS project. Finally, Rinaldi et al. use the PEST protocol implemented in iTOUGH2, combined with TOUGH-FLAC as the forward model, to perform inverse modeling for the In Salah CO2 demonstration site in Algeria. Using pore pressures and InSAR displacements as observations, this approach allows for a multi-physics joint inversion and takes advantage of the inverse capabilities of iTOUGH2 (parameter estimation, sensitivity analysis, uncertainty estimation …), applied to coupled fluid flow and geomechanics. On behalf of the members of the TOUGH Symposium 2015 Organizing Committee, I hope that this collection of papers will provide the readers of Computers & Geosciences with a recent view of the capabilities of the TOUGH family of codes, as well as with current modeling research domains in the geosciences. Last but not least, I would like to express my gratitude to all the reviewers who kindly contributed to this Special Issue, as well as the Journal staff who have made this publication possible.

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2026

Online LIBS–ML Framework for Dynamic Characterization of Heterogeneous Waste-Derived Gasification Feedstocks

LIBS−ML framework for real time feedstock characterization during continuous conveyor transport Heterogeneous waste derived feedstocks (e.g., waste coal, biomass and blends) introduce rapid variability in heating value and ash chemistry that affect gasifier operation, yet conventional laboratory characterization techniques are too slow to support proactive control. To address this gap, this study reports on an online, in situ, dynamic characterization framework that couple’s laser-induced breakdown spectroscopy (LIBS) with leakage safe machine learning (ML) regression to deliver real time, decision quality predictions of gasifier relevant properties. A controlled sample matrix spanning two different waste coals, two different biomasses, and engineered blends under two particle size conditions were constructed and benchmarked using standardized laboratory analyses for proximate/ultimate properties and ash composition. LIBS spectra were acquired dynamically as material flowed on a conveyor belt, using high energy 1064 nm laser ablation and shot averaging to improve repeatability and precision. Supervised regression models (multi layer perceptron (MLP) /artificial neural network (ANN), random forest (RF), and support vector regression (SVR)) and an optimized weighted ensemble were trained on emission line feature sets using nested cross validation with Bayesian hyperparameter tuning and validated against an independent hold out set. The proposed LIBS−ML workflow achieves near laboratory predictive fidelity across parametric targets (including higher heating value (HHV), ash content, fixed carbon, sulfur, major ash forming oxides, and initial deformation temperature (IDT)), with the weighted ensemble providing a robust default predictor under dynamic measurement conditions. These results demonstrate a practical pathway for real time feedstock characterization that can enable feedforward adjustments and more resilient gasifier operation for variable quality waste derived fuels.

Biomass↗
DOE OSTI2021

Adaptive hyperparameter updating for training restricted Boltzmann machines on quantum annealers

Restricted Boltzmann Machines (RBMs) have been proposed for developing neural networks for a variety of unsupervised machine learning applications such as image recognition, drug discovery, and materials design. The Boltzmann probability distribution is used as a model to identify network parameters by optimizing the likelihood of predicting an output given hidden states trained on available data. Training such networks often requires sampling over a large probability space that must be approximated during gradient based optimization. Quantum annealing has been proposed as a means to search this space more efficiently which has been experimentally investigated on D-Wave hardware. D-Wave implementation requires selection of an effective inverse temperature or hyperparameter (β) within the Boltzmann distribution which can strongly influence optimization. Here, we show how this parameter can be estimated as a hyperparameter applied to D-Wave hardware during neural network training by maximizing the likelihood or minimizing the Shannon entropy. We find both methods improve training RBMs based upon D-Wave hardware experimental validation on an image recognition problem. Neural network image reconstruction errors are evaluated using Bayesian uncertainty analysis which illustrate more than an order magnitude lower image reconstruction error using the maximum likelihood over manually optimizing the hyperparameter. The maximum likelihood method is also shown to out-perform minimizing the Shannon entropy for image reconstruction.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

From Optimization to Sampling Through Gradient Flows

Optimization and sampling algorithms play a central role in science and engineering as they enable finding optimal predictions, policies, and recommendations, as well as expected and equilibrium states of complex systems. The notion of “optimality” is formalized by the choice of an objective function, while the notion of an “expected” state is specified by a probabilistic model for the distribution of states. Optimizing rugged objective functions and sampling multimodal distributions is computationally challenging, especially in high-dimensional problems. Here, for this reason, many optimization and sampling methods have been developed by researchers working in disparate fields such as Bayesian statistics, molecular dynamics, genetics, quantum chemistry, machine learning, weather forecasting, econometrics, and medical imaging.

Trillos, N. García↗
DOE OSTI2016

ZEM: Integrated Framework for Real-Time Data and Model Analyses for Robust Environmental Management Decision Making - 16438

Typically environmental management (EM) problems require analysis of large and complex data sets originating from concurrent data streams with different data collection frequencies and pedigree. These big data sets require on-the-fly analysis, assimilation and integration into a series of models with different complexity for various types of model-based analyses. This is needed to provide fast iterative model analyses based on the latest available data to guide decision-making. Furthermore, the data and model are associated with uncertainties. The uncertainties are probabilistic (e.g. measurement errors) and non-probabilistic (unknowns, e.g. alternative conceptual models characterizing site conditions). To address all of these issues, we have developed an integrated framework for real-time data and model analyses for environmental decision-making called ZEM. The framework allows for seamless and on-the-fly integration of data and modeling results for robust and scientifically-defensible decision-making applying advanced decision analyses tools such as Bayesian-Information-Gap Decision Theory (BIG-DT). The framework also includes advanced methods for optimization that are capable of dealing with a large number of unknown model parameters, and surrogate (reduced order) modeling capabilities based on support vector regression techniques. The framework is predominantly coded in Julia, a state-of-the-art high-performance programing language (http://julialang.org) and incorporates the MADS computational tools (Model Analysis and Decision Support; http://mads.lanl.gov). We demonstrate the application of ZEM for data and model analyses at the LANL Chromium site. The ZEM framework will be released as open-source code and can be applied to any environmental management site. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2022

GIGA-Lens: Fast Bayesian Inference for Strong Gravitational Lens Modeling

We present GIGA-Lens: a gradient-informed, GPU-accelerated Bayesian framework for modeling strong gravitational lensing systems, implemented in TensorFlow and JAX. The three components, optimization using multistart gradient descent, posterior covariance estimation with variational inference, and sampling via Hamiltonian Monte Carlo, all take advantage of gradient information through automatic differentiation and massive parallelization on graphics processing units (GPUs). We test our pipeline on a large set of simulated systems and demonstrate in detail its high level of performance. The average time to model a single system on four Nvidia A100 GPUs is 105 s. The robustness, speed, and scalability offered by this framework make it possible to model the large number of strong lenses found in current surveys and present a very promising prospect for the modeling of ${ \mathcal O }({10}^{5})$ lensing systems expected to be discovered in the era of the Vera C. Rubin Observatory, Euclid, and the Nancy Grace Roman Space Telescope.

79 ASTRONOMY AND ASTROPHYSICS↗