Engineering Papers⌕ Search

SEARCH · Engineering Papers

Results for “Bayesian learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Targeted materials discovery using Bayesian algorithm execution

Rapid discovery and synthesis of future materials requires intelligent data acquisition strategies to navigate large design spaces. A popular strategy is Bayesian optimization, which aims to find candidates that maximize material properties; however, materials design often requires finding specific subsets of the design space which meet more complex or specialized goals. We present a framework that captures experimental goals through straightforward user-defined filtering algorithms. These algorithms are automatically translated into one of three intelligent, parameter-free, sequential data collection strategies (SwitchBAX, InfoBAX, and MeanBAX), bypassing the time-consuming and difficult process of task-specific acquisition function design. Our framework is tailored for typical discrete search spaces involving multiple measured physical properties and short time-horizon decision making. We demonstrate this approach on datasets for TiO 2 nanoparticle synthesis and magnetic materials characterization, and show that our methods are significantly more efficient than state-of-the-art approaches. Overall, our framework provides a practical solution for navigating the complexities of materials design, and helps lay groundwork for the accelerated development of advanced materials.

42 ENGINEERING↗
DOE OSTI2025

Active oversight and quality control in standard Bayesian optimization for autonomous experiments

The fusion of experimental automation and machine learning has catalyzed a new era in materials research, prominently featuring Gaussian Process (GP) Bayesian Optimization (BO) driven autonomous experiments. Here we introduce a Dual-GP approach that enhances traditional GPBO by adding a secondary surrogate model to dynamically constrain the experimental space based on real-time assessments of the raw experimental data. This Dual-GP approach enhances the optimization efficiency of traditional GPBO by isolating more promising space for BO sampling and more valuable experimental data for primary GP training. We also incorporate a flexible, human-in-the-loop intervention method in the Dual-GP workflow to adjust for unanticipated results. We demonstrate the effectiveness of the Dual-GP model with synthetic model data and implement this approach in autonomous pulsed laser deposition experimental data. This Dual-GP approach has broad applicability in diverse GPBO-driven experimental settings, providing a more adaptable and precise framework for refining autonomous experimentation for more efficient optimization.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Thinking Bayesian for plasma physicists

Bayesian statistics offers a powerful technique for plasma physicists to infer knowledge from the heterogeneous data types encountered. To explain this power, a simple example, Gaussian Process Regression, and the application of Bayesian statistics to inverse problems are explained. The likelihood is the key distribution because it contains the data model, or theoretic predictions, of the desired quantities. By using prior knowledge, the distribution of the inferred quantities of interest based on the data given can be inferred. Because it is a distribution of inferred quantities given the data and not a single prediction, uncertainty quantification is a natural consequence of Bayesian statistics. The benefits of machine learning in developing surrogate models for solving inverse problems are discussed, as well as progress in quantitatively understanding the errors that such a model introduces.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Calibrating hypersonic turbulence flow models with the HIFiRE-1 experiment using data-driven machine-learned models.

In this paper we study the efficacy of combining machine-learning methods with projection-based model reduction techniques for creating data-driven surrogate models of computationally expensive, high-fidelity physics models. Such surrogate models are essential for many-query applications e.g., engineering design optimization and parameter estimation, where it is necessary to invoke the high-fidelity model sequentially, many times. Surrogate models are usually constructed for individual scalar quantities. However there are scenarios where a spatially varying field needs to be modeled as a function of the model’s input parameters. Here we develop a method to do so, using projections to represent spatial variability while a machine-learned model captures the dependence of the model’s response on the inputs. The method is demonstrated on modeling the heat flux and pressure on the surface of the HIFiRE-1 geometry in a Mach 7.16 turbulent flow. The surrogate model is then used to perform Bayesian estimation of freestream conditions and parameters of the SST (Shear Stress Transport) turbulence model embedded in the high-fidelity (Reynolds-Averaged Navier–Stokes) flow simulator, using shock-tunnel data. The paper provides the first-ever Bayesian calibration of a turbulence model for complex hypersonic turbulent flows. We find that the primary issues in estimating the SST model parameters are the limited information content of the heat flux and pressure measurements and the large model-form error encountered in a certain part of the flow.

42 ENGINEERING↗
DOE OSTI2026

A score-based diffusion model approach for adaptive learning of stochastic partial differential equation solutions

In this paper, we propose a novel framework for adaptively learning the time-evolving solutions of stochastic partial differential equations (SPDEs) using score-based diffusion models within a recursive Bayesian inference setting. SPDEs play a central role in modeling complex physical systems under uncertainty, but their numerical solutions often suffer from model errors and reduced accuracy due to incomplete physical knowledge and environmental variability. To address these challenges, we encode the governing physics into the score function of a diffusion model using simulation data and incorporate observational information via a likelihood-based correction in a reverse-time stochastic differential equation. This enables adaptive learning through iterative refinement of the solution as new data becomes available. To improve computational efficiency in high-dimensional settings, we introduce the ensemble score filter, a training-free approximation of the score function designed for real-time inference. Numerical experiments on benchmark SPDEs demonstrate the accuracy and robustness of the proposed method under sparse and noisy observations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Machine learning for continuous quantum error correction on superconducting qubits

Abstract Continuous quantum error correction has been found to have certain advantages over discrete quantum error correction, such as a reduction in hardware resources and the elimination of error mechanisms introduced by having entangling gates and ancilla qubits. We propose a machine learning algorithm for continuous quantum error correction that is based on the use of a recurrent neural network to identify bit-flip errors from continuous noisy syndrome measurements. The algorithm is designed to operate on measurement signals deviating from the ideal behavior in which the mean value corresponds to a code syndrome value and the measurement has white noise. We analyze continuous measurements taken from a superconducting architecture using three transmon qubits to identify three significant practical examples of non-ideal behavior, namely auto-correlation at temporal short lags, transient syndrome dynamics after each bit-flip, and drift in the steady-state syndrome values over the course of many experiments. Based on these real-world imperfections, we generate synthetic measurement signals from which to train the recurrent neural network, and then test its proficiency when implementing active error correction, comparing this with a traditional double threshold scheme and a discrete Bayesian classifier. The results show that our machine learning protocol is able to outperform the double threshold protocol across all tests, achieving a final state fidelity comparable to the discrete Bayesian classifier.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2021

Discovery of Energy Storage Molecular Materials Using Quantum Chemistry-Guided Multiobjective Bayesian Optimization

Redox flow batteries (RFBs) are a promising technology for stationary energy storage applications due to their flexible design, scalability, and low cost. In RFBs, energy is carried in flowable redoxactive materials (redoxmers) which are stored externally and pumped to the cell during operation. Further improvements in the energy density of RFBs necessitates redoxmer designs with wider redox potential windows and higher solubility. Additionally, designing redoxmers with a fluorescence-enabled self-reporting functionality allows monitoring of the state of health of RFBs. To accelerate the discovery of redoxmers with desired properties, state-of-the- art machine learning ( ML) methods, such as multiobjective Bayesian optimization (MBO), are useful. Here, we first employed density functional theory calculations to generate a database of reduction potentials, solvation free energies, and absorption wavelengths for 1400 redoxmer molecules based on a 2,1,3-benzothiadiazole (BzNSN) core structure. From the computed properties, we identified 22 Pareto-optimal molecules that represent best trade-off among all of the desired properties. We further utilized these data to develop and benchmark an MBO approach to identify candidates quickly and efficiently with multiple targeted properties. With MBO, optimal candidates from the 1400-molecule data set can be identified at least 15 times more efficiently compared to the brute force or random selection approach. Importantly, we utilized this approach for discovering promising redoxmers from an unseen database of 1 million BzNSN-based molecules, where we discovered 16 new Pareto-optimal molecules with significant improvements in properties over the initial 1400 molecules. Furthermore, we anticipate that this active learning technique is general and can be utilized for the discovery of any class of functional materials that satisfies multiple desired property criteria.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Machine Learning Accelerates Innovation in Perovskite Manufacturing Scale-up (Final Technical Report (FTR))

We propose to address the challenge of the vast parameter space associated with perovskite manufacturing optimization, by developing a machine learning (ML)-assisted optimization framework for a scalable perovskite PV manufacturing tool. This framework will be interpretable, sequential, and rapidly adaptable to upgraded systems (e.g., via transfer learning). The tool is an open-air rapid spray plasma process (RSPP) of perovskite films, which has already been established at Stanford and is a unique platform to test and deploy the proposed ML-guided framework because the RSPP technique is able to conduct optimization experiments with a high throughput, and easily adjust a wide range of process variables.

14 SOLAR ENERGY↗
DOE OSTI2024

Tokamak divertor plasma emulation with machine learning

Abstract Future tokamak devices that aim to create conditions relevant to power plant operations must consider strategies for mitigating damage to plasma facing components in the divertor. One of the goals of MAST-U tokamak operations is to inform these considerations by researching advanced divertor configurations that aid stable plasma detachment. Machine design, scenario planning and detachment control would all greatly benefit from tools that enable rapid calculation of scenario-relevant quantities given some input parameters. This paper presents a method for generating large, simulated scrape-off layer data sets, which was applied to generate a data set of steady-state Hermes-3 simulations of the MAST-U tokamak. A machine learning model was constructed using a Bayesian approach to hyperparameter optimisation to predict diagnosable output quantities given control-relevant input features. The resulting best-performing model, which is based on a feedforward neural network, achieves high accuracy when predicting electron temperature at the divertor target and carbon impurity radiation front position and runs in around 1 ms in inference mode. Techniques for interpreting the predictions made by the model were applied, and a high-resolution parameter scan of upstream conditions was performed to demonstrate the utility of rapidly generating accurate predictions using the emulator. This work represents a step forward in the design of machine learning-driven emulators of tokamak exhaust simulation codes in operational modes relevant to divertor detachment control and plasma scenario design.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Machine Learning on a Robotic Platform for the Design of Polymer–Protein Hybrids

Polymer–protein hybrids are intriguing materials that can bolster protein stability in non-native environments, thereby enhancing their utility in diverse medicinal, commercial, and industrial applications. One stabilization strategy involves designing synthetic random copolymers with compositions attuned to the protein surface, but rational design is complicated by the vast chemical and composition space. Here, a strategy is reported to design protein-stabilizing copolymers based on active machine learning, facilitated by automated material synthesis and characterization platforms. The versatility and robustness of the approach is demonstrated by the successful identification of copolymers that preserve, or even enhance, the activity of three chemically distinct enzymes following exposure to thermal denaturing conditions. Although systematic screening results in mixed success, active learning appropriately identifies unique and effective copolymer chemistries for the stabilization of each enzyme. Overall, this work broadens the capabilities to design fit-for-purpose synthetic copolymers that promote or otherwise manipulate protein activity, with extensions toward the design of robust polymer–protein hybrid materials.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Structural mode coupling in perovskite oxides using hypothesis-driven active learning

Abstract Finding the ground-state structure with minimum energy is paramount to designing any material. In ABO 3 -type perovskite oxides with Pnma symmetry, the lowest energy phase is driven by an inherent trilinear coupling between the two primary order parameters such as rotation and tilt with antiferroelectric displacement of the A-site cations as established via hybrid improper ferroelectric mechanism. Conventionally, finding the relevant mode coupling driving phase transition requires performing first-principles calculations which is computationally time-consuming as well as expensive. It involves following an intuitive iterative hit and trial method of (a) adding two or multiple mode vectors, followed by (b) evaluating which combination would lead to the ground-state energy. In this study, we show how a hypothesis-driven active learning framework can identify suitable mode couplings within the Landau free energy expansion with minimal information on amplitudes of modes for a series of double perovskite oxides with A-site layered, columnar and rocksalt ordering. This scheme is expected to be applicable universally for understanding atomistic mechanisms derived from various structural mode couplings behind functionalities, for e.g. polarization, magnetization and metal–insulator transitions.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Neural simulation-based inference of the neutron star equation of state directly from telescope spectra

Neutron stars provide a unique opportunity to study strongly interacting matter under extreme density conditions. The intricacies of matter inside neutron stars and their equation of state are not directly visible, but determine bulk properties, such as mass and radius, which affect the star's thermal X-ray emissions. However, the telescope spectra of these emissions are also affected by the stellar distance, hydrogen column, and effective surface temperature, which are not always well-constrained. Uncertainties on these nuisance parameters must be accounted for when making a robust estimation of the equation of state. In this study, we develop a novel methodology that, for the first time, can infer the full posterior distribution of both the equation of state and nuisance parameters directly from telescope observations. This method relies on the use of neural likelihood estimation, in which normalizing flows use samples of simulated telescope data to learn the likelihood of the neutron star spectra as a function of these parameters, coupled with Hamiltonian Monte Carlo methods to efficiently sample from the corresponding posterior distribution. Our approach surpasses the accuracy of previous methods, improves the interpretability of the results by providing access to the full posterior distribution, and naturally scales to a growing number of neutron star observations expected in the coming years.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

Deducing neutron star equation of state from telescope spectra with machine-learning-derived likelihoods

The interiors of neutron stars reach densities and temperatures beyond the limits of terrestrial experiments, providing vital laboratories for probing nuclear physics. While the star's interior is not directly observable, its pressure and density determine the star's macroscopic structure which affects the spectra observed in telescopes. The relationship between the observations and the internal state is complex and partially intractable, presenting difficulties for inference. Previous work has focused on the regression from stellar spectra of parameters describing the internal state. We demonstrate a calculation of the full likelihood of the internal state parameters given observations, accomplished by replacing intractable elements with machine learning models trained on samples of simulated stars. Our machine-learning-derived likelihood allows us to perform maximum a posteriori estimation of the parameters of interest, as well as full scans. We demonstrate the technique by inferring stellar mass and radius from an individual stellar spectrum, as well as equation of state parameters from a set of spectra. Our results are more precise than pure regression models, reducing the width of the parameter residuals by 11.8% in the most realistic scenario. The neural networks will be released as a tool for fast simulation of neutron star properties and observed spectra.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

A machine learning Automated Recommendation Tool for synthetic biology

Abstract Synthetic biology allows us to bioengineer cells to synthesize novel valuable molecules such as renewable biofuels or anticancer drugs. However, traditional synthetic biology approaches involve ad-hoc engineering practices, which lead to long development times. Here, we present the Automated Recommendation Tool (ART), a tool that leverages machine learning and probabilistic modeling techniques to guide synthetic biology in a systematic fashion, without the need for a full mechanistic understanding of the biological system. Using sampling-based optimization, ART provides a set of recommended strains to be built in the next engineering cycle, alongside probabilistic predictions of their production levels. We demonstrate the capabilities of ART on simulated data sets, as well as experimental data from real metabolic engineering projects producing renewable biofuels, hoppy flavored beer without hops, fatty acids, and tryptophan. Finally, we discuss the limitations of this approach, and the practical consequences of the underlying assumptions failing.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Particle image velocimetry analysis with simultaneous uncertainty quantification using Bayesian neural networks

Particle image velocimetry (PIV) is an effective tool in experimental fluid mechanics for extracting flow fields from images. Recently, convolutional neural networks (CNNs) have been used to perform PIV analysis with accuracy on par with classical methods. Here we extend the use of CNNs to analyze PIV data while providing simultaneous uncertainty quantification on the inferred flow field. The method we apply in this paper is a Bayesian convolutional neural network (BCNN) which learns distributions of the CNN weights through variational Bayes. In order to demonstrate the utility of BCNNs for the PIV task, we compare the performance of three distinct BCNN models with simple architectures. The first network estimates flow velocity from image interrogation regions only. Our second model learns to infer velocity from both the image interrogation regions and interrogation region cross-correlation maps. Finally, our best performing network infers velocities from interrogation region cross-correlation maps only. We find that BCNNs using interrogation region cross-correlation maps as inputs perform better than those using interrogation windows only as inputs and discuss reasons why this may be the case. Additionally, we test the best performing BCNN on a full synthetic test image pair and a real image pair from the 1st International PIV Challenge. We show that ~98% of true particle displacements from the full synthetic image pair can be captured within the BCNN's 95% confidence intervals, and that the BCNN's performance on the real image pair is quantitatively similar to that of algorithms tested in the 1st International PIV Challenge. Finally, we show that BCNNs can be generalized to be used with multi-pass PIV algorithms with a moderate loss in accuracy, which may be overcome by future work on finetuning and training schemes. So to our knowledge, this is the first use of Bayesian neural networks to perform PIV.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Bayesian Analysis of TRISO Fuel: Quantifying Model Inadequacy, Incorporating Lower-Length-Scale Effects, and Developing Parallel Active Learning Capabilities

The U.S. Department of Energy (DOE)’s Nuclear Energy Advanced Modeling and Simulation (NEAMS) program aims to develop predictive capabilities by applying computational methods to the analysis and design of advanced reactor and fuel-cycle systems. This program has been providing engineering-scale support for the continued development of BISON, a high-fidelity, high-resolution fuel performance tool. Fuel behavior in nuclear reactors is governed by a complex network of mechanisms that interact with various other physics aspects in the reactor system. Any model developed to represent fuel behavior will likely be idealized, resulting in uncertainties when comparing their predictions against the observed data. In Fiscal Year (FY)-23, we initiated the Uncertainty Quantification (UQ) work by using Bayesian methods to establish a level of model trustworthiness and further improve it, with a particular emphasis on TRI-Structural isOtropic (TRISO) nuclear fuel. This year, we further expanded on that UQ work by investigating an approach to quantifying model inadequacy and accounting for lower-length scale (LLS) effects in TRISO silver (Ag) release modeling. Furthermore, we are implementing parallel active learning capabilities to reduce the computational cost (i.e., required computational resources and elapsed time) of performing UQ. Specifically, we utilized The Kennedy O’Hagan framework for Bayesian uncertainty quantification (KOH) to account for model inadequacy in TRISO Ag release predictions made by BISON. The KOH framework represents an improvement over the standard Bayesian framework used in FY-23. Explicitly accounting for model inadequacy in the Bayesian framework helps establish the level of experimental noise uncertainty in the Advanced Gas Reactor (AGR) data. We compared the inverse UQ results obtained from both the standard Bayesian and KOH frameworks in light of the AGR-2/3/4 data, and also compared the predictive UQ results obtained from these two frameworks in light of the AGR-1 data. Next, we investigated the impact of considering LLS effects in the Ag release simulations. We developed an expanded database of LLS simulated effective diffusivities for Ag, covering a wide range of microstructures and temperatures. Using this database, we developed a framework for incorporating LLS effects into the engineering-scale Ag release UQ. We developed both parametric and non-parametric approaches for bridging the length scales. We then investigated the inverse UQ results in light of the AGR-2/3/4 data and the predictive UQ results in light of the AGR-1 data, and compared the LLS-informed approach and the Arrhenius equation, which does not include microstructure information. Finally, we discussed implementing parallel active learning capabilities in the Multiphysics Object Oriented Simulation Environment (MOOSE)/BISON to reduce the computational cost (i.e., computational resources and elapsed time) of Bayesian UQ. For verification purposes, we first tested these new capabil ities on a species interaction problem. We then demonstrated them on the TRISO Ag release application, showing that parallel active learning capabilities can enhance the accuracy of UQ while also substantially reducing the computational cost in comparison to the reference methods developed in FY-23.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

MOOSE ProbML: Parallelizable Probabilistic Machine Learning and Uncertainty Quantification Capabilities

The Multiphysics Object Oriented Simulation Environment (MOOSE) is a widely used open- source finite element software for performing multiphysics multiscale simulations in a massively parallel fashion. Recently, the computational team at Idaho National Laboratory (INL) has implemented Probabilistic Machine Learning (ProbML) capabilities in MOOSE—in a parallelized fashion—and enable active learning with large-scale computational models for tasks such as surrogate model development, scale bridging, forward/inverse uncertainty quantification (UQ), Bayesian optimization, etc. This presentation summarizes these developments in MOOSE along with demonstrations on several real applications relevant to nuclear energy. At the fundamental level, samplers like Monte Carlo/Latin Hypercube, variance reduction, parallelized Markov Chain Monte Carlo (MCMC) support uncertainty propagation in both forward and inverse settings. These samplers can be integrated with the Gaussian processes (GP) suite in MOOSE, which offer several variants like scalar GPs, multi-output GPs, and deep GPs, to enable active learning. These GPs can be tuned using gradient-based optimization methods like Adam and its variants or gradient-free methods like the elliptical slice sampler (a variant of MCMC adept under Gaussian settings) for more complex covariance kernels or likelihoods whose gradient computations can be cumbersome. A variety of batch acquisition functions permit parallelized evaluation of the computational model and support different learning objectives with high efficiency like Bayesian inference, global surrogate development, optimization, etc. Furthermore, libtorch integration supports training, evaluation, and re-training of neural networks and other complex machine learning models in active learning settings. The impacts of these developments are shown on several real applications: (1) nuclear fuel inverse UQ and model inadequacy assessment using the Kennedy O’Hagan framework; (2) uncertainty aware surrogate modeling for additive manufacturing to predict field quantities; (3) nuclear reactor rare events analysis; and (4) complex fluid flow prediction using a global surrogate with quantified prediction uncertainty. Finally, the outlook of MOOSE ProbML is discussed for both outer-loop and inner-loop computations in the broad view to accelerate fuels and materials qualification, address gaps in knowledge and data, and assess new reactor/fuel systems.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗